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CAMERA CALIBRATING APPARATUS 



TECHNICAL FIELD OF THE INVENTION 

rOOOll The present invention relates to a camera calibrating apparatus, and more particularly to a 
camera calibrating apparatus for calibrating a camera mounted on an automotive vehicle or the 
like. 

DESCRIPTION OF THE RELATED ART 

[00021 In recent years, there has been developed an imaging controlling apparatus, for example 
an electronic control unit (herein after simply referred to as "ECU"), which is used to be 
electrically connected to a camera mounted on an exterior surface of an automotive vehicle to 
detect the position occupied by an object located on a road, m particular, in the vicinity of the 
automotive vehicle on the basis of image information obtained by the camera. When, in general, 
the camera is mounted on the automotive vehicle in combination with the imaging controlling 
apparatus of this type, the calibration of the camera is performed by a camera calibrating 
apparatus in order to specify optical parameters of the camera. 

[00031 Up xmtil now, there have been proposed a wide variety of camera calibrating apparatuses 
of this type, all of which are classified into two different types includmg a furst type to calibrate 
the camera before that camera is mounted on the automotive vehicle, and a second type to 
calibrate the camera after that camera is mounted on the automotive vehicle. For the purpose of 
facilitating an assembly work of the camera to the automotive vehicle, there is a growing demand 
for the camera calibrating apparatus for calibrating the camera mounted on the automotive 
vehicle in recent years. 

r00041 As shown in FIGS. 39 to 41, the conventional camera calibrating apparatus 500 is 
electrically connected to the camera 510 as an imaging device. The camera 510 includes a 
housing unit 511 and an optical section 512 supported by the housing unit 51 1 to obtain image 
information through the optical section 512. 
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[00051 The conventional camera calibrating apparatus 500 comprises a first housing position 
information storing s e ction 515 for storing first housing position information indioativo of th e 
position occupi e d by th e housing unit 511 in th e first coordination systom 501 of a cam e ra 
producing plant, and a s e cond housing position information storing section 516 for stonng 
second housing position information mdioative of the position oooupiod by the housing unit 511 
in th o r '= >^nn^ nnnrHipntinn nyntflm SO? nf nn niitnmntivQ vohiole producing plant a first housing 
position information storing section 515 for storing first housing position information of the 
housing unit 551 obtained on the basis of the first coordinate system 501 defined in a camera 
producing plant and a second housing position information storing section 516 for storing 
second housing position information of the housing unit 551 obtained on the basis of the second 
coordinate system 502 defined in an automotive vehicle producing plant . 
r00061 The camera calibrating apparatus 500 is adapted to revise the camera 510 in the camera 
producmg plant by using a revising marker 505 located in the first coordination coordinate 
system 501. Here, the term "the calibration of the camera 510" is intended to indicate the 
calculation of the accurate position of the optical section 512 of the camera 510 mounted on the 
automotive vehicle 508. 

r00071 The conventional camera calibrating apparatus 500 further comprises a first optical 
position information producing section 517 for producing first optical position information 
indicative of the position of the optical section 512 in the fu:st coordination coordinate system 
501, and a first optical position information storing section 518 for storing the first optical 
position information produced by the furst optical position information producing section 517. 
The first optical position information producing section 517 is adapted to produce optical 
position information indicative of the position of the optical section 512 in the first coordination 
coordinate system 501 on the basis of the image information of the revising marker 505 obtained 
by the camera 510. 

rOOOSI The conventional camera calibrating apparatus 500 fiirther comprises a second optical 
position information producing section 520 for producing second optical information indicative 
of the position of the optical section 512 in the second coordination coordinate system 502, and a 
second optical position information storing section 530 for storing the second optical position 
information produced by the second optical position information producing section 520. 
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r00091 The second optical position infonnation producing section 520 is adapted to produce 
optical position information indicative of the position of the optical section 512 in the second 
coordination coordinate system 502 from the second housing position information stored m the 
second housing position information storing section 516 on the basis of the first housing position 
information stored in the first housing position information storing section 515 and the first 
optical position information stored in the first optical position information storing section 518. 
rOOlOl The conventional camera calibrating apparatus 500 thus constructed as previously 
mentioned is adapted to revise the camera in the camera producing plant by calculating the 
position of the optical section 512 in the second coordination coordinate system 502, while the 
image controlUng apparatus is adapted to detect the position occupied by an object located on the 
road through the image information obtained by the camera 510 mounted on the automotive 
vehicle 508. 

fOOll] The conventional camera calibrating apparatus as previously mentioned, however, 
encounters such a problem that the second optical position information stored in the second 
optical position information storing section is not calibrated. This leads to the fact that the 
position of the object located on the road caimot be accurately detected by the image controlling 
apparatus if the camera is mounted on an inaccurate position of the automotive vehicle. 
T00121 It is, therefore, an object of the present invention to provide a camera calibrating 
apparatus which can calibrate optical parameters of the optical section of the camera mounted on 
the automotive vehicle or the like. 

DISCLOSURE OF THE INVENTION 

[00131 In accordance with the present invention, there is provided a camera calibrating apparatus 
to be operative in combination with an imaging device which includes a housing unit and an 
optical section supported by the housing unit to obtain image information thorough the optical 
section, and adapted to calibrate position information of the optical section, comprising: first 
housing position information storing means for storing first housing position information 
indicative of the position of the housing unit in the first coordination coordinate system; second 
housing position information storing means for storing second housing position information 
indicative of the position of the housing unit in the second coordination coordinate system; fu-st 
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optical position information producing means for producing first optical position information 
indicative of the position of the optical section in the first coordination coordinate system on the 
basis of the image information obtained by tiie imaging device in the first coordination 
coordinate system; first optical position information storing means for storing the first optical 
position information produced by the first optical position information producing means; second 
optical position information producing means for producing second optical position information 
indicative of the position of the optical section in the second coordination coordinate system 
fi-om the second housing position information stored by the second housing position information 
storing means on the basis of the first housing position information stored by the first housing 
position information storing means and the first optical position information stored by the first 
optical position information storing means; second optical position information storing means for 
storing the second optical position information produced by the second optical position 
information producing means; and calibrating means for calibrating the second optical position 
information stored by the second optical position information storing means on the basis of the 
image information obtained by the imaging device in the second coordination coordinate system. 
The camera calibrating apparatus according to the present invention thus constructed as 
previously mentioned can calibrate parameters of the optical section mounted on an automotive 
vehicle or the like. 

[0014] In accordance with the present invention, there is provided a camera calibrating apparatus 
to be operative in combination with an imaging device which includes a housing unit and an 
optical section supported by the housing unit to obtain image information thorough the optical 
section, and adapted to calibrate optical position information about the position of the optical 
section, comprising: optical position information storing means for storing optical position 
information indicative of the position of the optical section to a predetermined coordinate 
system; and calibrating means for calibrating the optical position information stored by the 
optical position information storing means on the basis of the image information obtained by the 
imaging device on the predetermined coordinate system. The camera calibrating apparatus 
according to the present invention thus constructed as previously mentioned can calibrate 
parameters of the optical section mounted on an automotive vehicle or the like. 
[00151 In accordance with the present invention, there is provided a camera calibrating apparatus 
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to be operative in combination with an imaging device which includes a housing unit and an 
optical section supported by the housing unit to obtain image information thorough the optical 
section, and adapted to calibrate optical position information about the position of the optical 
section, comprising: first housing position information storing means for storing first housing 
position information indicative of the position of the housing unit in the first coordination 
coordinate system in which a revising marker is located; second housing position information 
storing means for storing second housing position information indicative of the position of the 
housing unit in the second coordmation coordinate system in which a calibrating marker is 
located; first optical position information producing means for producing first optical position 
information indicative of the position of the optical section in the first coordination coordmate 
system on the basis of the image information of the revising marker obtained by the imaging 
device; furst optical position information storing means for storing the first optical position 
information produced by the first optical position information producing means; second optical 
position information producing means for producing second optical position information 
indicative of tiie position of the optical section in the second coordination coordinate system 
firom the second housing position information stored by the second housing position information 
storing means on the basis of the first housing position information stored by the first housing 
position information storing means and the first optical position information stored by the first 
optical position information storing means; estimated location information producing means for 
producing estimated location information indicative of the position of the calibrating marker to 
an image coordination coordinate system of the imaging device on the basis of the second optical 
position information produced by the second optical position information producing means; 
second optical position information storing means for storing the second optical position 
information produced by the second optical position information producing means; estimated 
location information storing means for storing the estimated location information produced by 
the estimated location information estimating means; and calibrating means for calibrating tiie 
second optical position information stored by the second optical position information storing 
means on the basis of the unage information of the calibrating marker obtained by the imaging 
device and the estimated location information stored by the estimated location information 
storing means. The camera calibrating apparatus according to the present invention thus 
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constracted as previously mentioned can calibrate the second optical position information by 
using a simple calibrating marker. 

r00161 In the camera calibrating apparatus according to the present invention, the calibrating 
means includes: image location information extracting means for extracting an image location 
information indicative of an image location of the calibrating marker to the image coordination 
coordinate system of the imagjng device on the basis of the image information of the calibrating 
marker obtained by the imaging device; calibration value calculating means for calculating a 
calibration value of the second optical position information stored by the second optical position 
information storing means on the basis of the image location information extracted by the image 
location information extracting means and the estimated location information stored by the 
estimated location information storing means; and optical position information calibrating means 
for calibrating the second optical position information stored by the second optical position 
information storing means on the basis of the calibration value calculated by the calibration value 
calculating means. The camera calibrating apparatus according to the present invention thus 
constructed as previously mentioned can calculate calibration value of the second optical 
position information by using a simple calibrating marker. 

r00171 In the camera calibrating apparatus according to the present invention, the calibrating 
means is adapted to calibrate a deviation of a rotational component of the second optical position 
information. The camera calibrating apparatus according to the present mvention thus 
constructed as previously mentioned can calibrate the second optical position information by 
using a simple calibrating marker without taking into account parallel shift component of the 
second optical position information. 

[00181 In the camera calibratmg apparatus according to the present invention, the image location 
information extracting means includes: image displaying means for displaying an image of the 
calibrating marker obtained by the unaging device, and image location specifying means for 
specifying the image location of the calibrating marker on the image of the calibrating marker 
displayed by the image displaying means and to ensure that the image location information is 
extracted. The camera calibrating apparatus according to the present invention thus constructed 
as previously mentioned can specify the image location of the calibrating marker to ensure that 
the image location information of the calibrating marker is extracted. 
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[00 19] In the camera calibrating apparatus according to the present invention, the image location 
information extracting means includes: estimated area information storing means for storing 
estimated area information indicative of the calibrating marker to the image coordination 
coordinate system of the unaging device; and image location searching means for searching the 
image location of the calibrating marker from the image information of the calibrating marker 
obtained by the imaging device on the basis of the estimated area information stored by the 
estimated area information storing means and the estimated location information stored by the 
estimated location information storing means to ensure that the image location information is 
extracted. The camera calibrating apparatus according to the present invention thus constructed 
as previously mentioned can search the image location of the calibrating marker to ensure that 
the image location information of the calibrating marker is easily extracted. 
100201 In accordance with the present invention, there is provided a camera calibrating apparatus 
to be operative in combination with an imaging device which includes a housing unit and an 
optical section supported by the housing unit to obtain image information thorough the optical 
section, and adapted to calibrate optical position information about the position of the optical 
section, comprising: first housing position information storing means for storing first housing 
position information indicative of the position of the housing unit in the first coordination 
coordinate system in which a revising marker is located; second housing position information 
storing means for storing second housing position information indicative of the position of the 
housing unit in the second coordination coordinate system; first optical position information 
producing means for producing first optical position information indicative of the position of the 
optical section in the first coordination coordinate system on the basis of the image information 
obtained by the imaging device and indicative of the revising marker; first optical position 
information storing means for storing the first optical position mformation produced by the first 
optical position information producing means; second optical position information producing 
means for producing second optical position information indicative of the position of the optical 
section in the second coordination coordinate system from the second housing position 
information stored by the second housing position information storing means on the basis of the 
first housing position information stored by the first housing position information storing means 
and the furst optical position information stored by the first optical position information storing 
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means; second optical position information storing means for storing the second optical position 
information produced by the second optical position information producing means; and 
calibrating means for calibrating the second optical position information stored by the second 
optical position information storing means on the basis of a motion vector of the image 
information obtained by the imaging device in the second coordination coordinate system. The 
camera calibrating apparatus according to the present invention thus constructed £is previously 
mentioned can calibrate the second optical position information by using the motion vector. 
r00211 In the camera calibrating apparatus according to the present invention, the calibrating 
means includes: plane-projected image producing means for producing a plane-projected image 
from the image information obtained by the imaging device in the second coordination 
coordinate system; plane-projected image dividing means for dividing the plane-projected image 
produced by the plane-projected image producing means into a plurality of image segments; 
motion vector extracting means for extracting the motion vector from the image segments 
divided by the plane-projected image dividing means; calibration value calculating means for 
calculating a calibration value of the second optical position information stored by the second 
optical position information storing means on the basis of the motion vector extracted by the 
motion vector extracting means; and optical position information calibrating means for 
calibrating the second optical position information stored by the second optical position 
information storing means on the basis of the calibration value calculated by the calibration value 
calculating means. The camera calibrating apparatus according to the present invention thus 
constructed as previously mentioned can easily extract the motion vector from the image 
segments. 

[00221 In the camera calibrating apparatus according to the present invention, a dividing marker 
located in the second coordination coordinate system in a predetermined relationship with the 
position of the housing unit represented by the second position information stored by the second 
position information storing means. The plane-projected image dividing means is adapted to 
divide the plane-projected image produced by the plane-projected image producing means into a 
plurality of image segments on the basis of the image information of the dividing marker 
obtained by the imaging device. The camera calibrating apparatus according to the present 
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invention thus constructed as previously mentioned can accurately divide the plane-projected 
image by using the dividing marker. 

[00231 In accordance with the present mvention, there is provided a camera calibrating apparatus 
to be operative in combination with an imaging device which includes a housmg unit and an 
optical section supported by the housing unit to obtain image information thorough the optical 
section, and adapted to calibrate position information of the optical section, comprising: first 
housing position information storing means for storing first housing position information 
indicative of the position of the housing unit in the first coordination coordinate system in which 
a revising marker is located; second housing position information storing means for storing 
second housing position information indicative of the position of the housing unit in the second 
coordination coordinate system in which an automotive vehicle is located; first optical position 
information producing means for producing first optical position information indicative of the 
position of the optical section in the first coordination coordinate system on the basis of the 
image information of the revising marker obtained by the imaging device; first optical position 
information storing means for storing the first optical position information produced by the first 
optical position information producing means; second optical position information producing 
means for producing second optical position information indicative of the position of the optical 
section in the second coordination coordinate system from the second housing position 
information stored by the second housing position information storing means on the basis of the 
first housing position information stored by the first housing position information storing means 
and the first optical position information stored by the first optical position information storing 
means; estimated location information producing means for producing estimated location 
information indicative of the position of the automotive vehicle to an image coordination 
coordinate system of the imaging device on the basis of the second optical position information 
produced by the second optical position information producing means; second optical position 
information storing means for storing the second optical position information produced by the 
second optical position information producing means; estimated location mformation storing 
means for storing the estimated location information produced by the estimated location 
information estimating means; and calibrating means for calibrating the second optical position 
information stored by the second optical position information storing means on the basis of the 
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image information of the automotive vehicle obtained by the imaging device and the estimated 
location information stored by the estimated location information storing means. The camera 
calibrating apparatus according to tiie present invention thus constructed as previously 
mentioned can calibrate the second optical position information by using a portion of the 
automotive vehicle. 

r00241 In the camera calibrating apparatus according to the present invention, the calibrating 
means includes: image location information extracting means for extracting an image location 
information indicative of an image location of the automotive vehicle in the image coordination 
coordinate system of the imaging device on the basis of the image information of the automotive 
vehicle obtained by the imaging device; calibration value calculating means for calculating a 
calibration value of the second optical position information stored by the second optical position 
information storing means on the basis of the image location information extracted by the image 
location information extracting means and the estimated location information stored by the 
estimated location information storing means; and optical position information calibrating means 
for calibrating the second optical position information stored by the second optical position 
information storing means on the basis of the calibration value calculated by the calibration value 
calculating means. The camera calibrating apparatus according to the present invention thus 
constructed as previously mentioned can calculate the calibration value of the second optical 
position information by using a portion of the automotive vehicle. 

[00251 In the camera calibrating apparatus according to the present invention, the calibration 
value calculating means includes: superimposing means for superimposing a profile line of the 
automotive vehicle represented by the image location information on a profile line of the 
automotive vehicle represented by the estimated location information; extracting means for 
extracting a plurality of points from the overlapped profile lines of the automotive vehicle 
superimposed by the superimposing means, and calculating means for calculating a calibration 
value of the second optical position information by comparing the points of the image location 
information with the points of the estimated location information. The camera calibrating 
apparatus according to the present invention thus constructed as previously mentioned can 
extract the points from the profile line of the automotive vehicle to ensure that the second optical 
position information is extracted. 
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r00261 In the camera calibrating apparatus according to the present invention, the imaging device 
is mounted on an automotive vehicle. The camera calibrating apparatus according to the present 
invention thus constructed as previously mentioned can calibrate parameters of the optical 
section to accurately detect the position of an object located on the road even if the housing unit 
is mounted on the inaccurate position of the automotive vehicle. 

r00271 In accordance with the present mvention, there is provided an unaging system comprises 
a camera calibrating apparatus. The imaging system according to the present mvention thus 
constructed as previously mentioned can calibrate parameters of the optical section mounted on 
an automotive vehicle or the like. 

[00281 In accordance with the present invention, there is provided an imagmg control system 
comprises a camera calibrating apparatus. The unaging control system according to the present 
invention thus constructed as previously mentioned can calibrate parameters of the optical 
section mounted on an automotive vehicle or the like. 

[00291 In accordance with the present invention, there is provided a camera calibrating method 
of calibrating the position of an optical section supported by a housing unit of a camera for 
obtaining image information through the optical section, comprising: a first housing position 
information storing step of storing first housing position information indicative of the position of 
the housing unit in the first coordination coordinate system; a second housing position 
information storing step of storing second housing position information indicative of the position 
of the housing unit in the second coordination coordinate system; a first optical position 
information producmg step of producing first optical position information indicative of the 
position of the optical section in the first coordination coordinate system on the basis of the 
image information obtained by the imaging device in the first coordination coordinate system; a 
first optical position information storing step of storing the first optical position information 
produced in the first optical position infomiation producing step; a second optical position 
information producing step of producing second optical position information indicative of the 
position of the optical section in the second coordination coordinate system firom the second 
housing position information stored in the second housing position information storing step on 
the basis of the first housing position information stored in the fu-st housing position information 
storing step and the first optical position information stored in the first optical position 
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information storing step; a second optical position information storing step of storing the second 
optical position information produced in the second optical position information producmg step; 
and a calibrating step of calibrating the second optical position information stored in the second 
optical position information storing step on the basis of the image information obtained by the 
imaging device in the second coordination coordinate system. The camera calibrating method 
according to the present invention thus constructed as previously mentioned can calibrate 
parameters of the optical section mounted on an automotive vehicle or the like to accurately 
detect the position of an object located on the road. 

[00301 In accordance with the present invention, there is provided a camera calibrating method 
of calibrating the position of an optical section supported by a housing unit of a camera for 
obtaining unage information through the optical section, comprising: an optical position 
information storing step of storing optical position information indicative of the position of the 
optical section to a predetermmed coordinate system; and a calibrating step of calibrating the 
optical position information stored in the optical position information storing step on the basis of 
the image information obtained by the imaging device on the predetermined coordinate system. 
The camera calibrating method according to the present invention thus constructed as previously 
mentioned can calibrate parameters of the optical section mounted on an automotive vehicle or 
the like to accurately detect the position of an object located on the road. 
r003n In accordance with the present invention, there is provided a camera calibrating method 
of calibrating the position of an optical section supported by a housing unit of a camera for 
obtaining image information through the optical section, comprising: a first housing position 
information storing step of storing first housing position information indicative of the position of 
the housing unit in the first coordination coordinate system in which a revising marker is located; 
a second housing position information storing step of storing second housing position 
information indicative of the position of the housing unit m the second ooordinati(» coordinate 
system in which a calibrating marker is located; a first optical position information producing 
step of producing first optical position information indicative of the position of the optical 
section in the first coordination coordinate system on the basis of the image information of the 
revising marker obtained by the imaging device; a first optical position information storing step 
of storing the first optical position information produced m the first optical position information 
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producing step; a second optical position information producing step of producing second optical 
position information indicative of the position of the optical section in the second coordination 
coordinate system from the second housing position information stored in the second housing 
position information storing step on the basis of the first housing position information stored in 
the first housing position information storing step and the first optical position information stored 
in the first optical position information storing step; an estimated location information producing 
step of producing estimated location information indicative of the position of the calibrating 
marker to an image coordination coordinate system of the imaging device on the basis of the 
second optical position information produced in the second optical position information 
producing step; a second optical position information storing step of storing the second optical 
position information produced in the second optical position information producing step; an 
estimated location information storing step of storing the estimated location information 
produced in the estimated location information estimating step; and a calibrating step of 
calibrating the second optical position information stored in the second optical position 
information storing step on the basis of the image information of the calibrating marker obtained 
by the imaging device and the estimated location information stored in the estimated location 
information storing step. The camera calibrating method according to the present invention thus 
constructed as previously mentioned can calibrate the second optical position information by 
using the simple calibrating marker. 

f0032] In accordance with the present invention, there is provided a camera calibrating method 
of calibrating the position of an optical section supported by a housing unit of a camera for 
obtaining image information through the optical section, comprising: a first housing position 
information storing step of storing first housing position information indicative of the position of 
the housing unit in the first coordination coordinate system in which a revising marker is located; 
a second housing position information storing step of storing second housing position 
information indicative of the position of the housing unit in the second coordination coordinate 
system; a first optical position information producing step of producing first optical position 
information indicative of the position of the optical section in the first coordination coordinate 
system on the basis of the image information obtained by the imaging device and indicative of 
the revising marker; a first optical position information storing step of storing the first optical 
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position infonnation produced in the first optical position information producing step; a second 
optical position information producing step of producing second optical position information 
indicative of the position of the optical section in the second coordination coordinate system 
from the second housing position information stored in the second housing position information 
storing step on the basis of the first housing position information stored in the first housing 
position information storing step and the first optical position information stored in the fu-st 
optical position information storing step; a second optical position information storing step of 
storing die second optical position information produced in the second optical position 
information producing step; and a calibrating step of calibrating the second optical position 
information stored in the second optical position information storing step on the basis of a 
motion vector of the image information obtained by the imaging device in the second 
coordination coordinate system. The camera calibrating method according to the present 
invention thus constructed as previously mentioned can calibrate the second optical position 
information by using the motion vector. 

[00331 In accordance with the present invention, there is provided a camera calibratmg method 
of calibrating the position of an optical section supported by a housing unit of a camera for 
obtaining image information through the optical section, comprising: a first housing position 
information storing step of storing first housing position information indicative of the position of 
the housing unit in the first coordination coordinate system in which a revising marker is located; 
a second housing position information storing step of storing second housing position 
information indicative of the position of the housing unit in the second coordination coordinate 
system in which an automotive vehicle is located; a fu-st optical position information producing 
step of producing first optical position information indicative of the position of the optical 
section in the first coordination coordinate system on the basis of the image information of the 
revising marker obtained by the imaging device; a first optical position information storing step 
of storing the first optical position information produced in the furst optical position mformation 
producing step; a second optical position information producing step of producing second optical 
position information indicative of the position of the optical section in the second coordination 
coordinate system from the second housing position information stored in the second housing 
position information storing step on the basis of the first housing position information stored in 
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the first housing position information storing step and the first optical position information stored 
in the first optical position information storing step; an estimated location information producing 
step of producing estimated location information indicative of the position of the automotive 
vehicle to an image coordination coordinate system of the imaging device on the basis of the 
second optical position information produced in the second optical position information 
producing step; a second optical position information storing step of storing the second optical 
position information produced in the second optical position information producing step; an 
estimated location infomiation storing step of storing the estimated location information 
produced in the estimated location information estimating step; and a calibrating step of 
calibrating the second optical position information stored in the second optical position 
information storing step on the basis of the image information of the automotive vehicle obtained 
by the imaging device and the estimated location information stored in the estimated location 
information storing step. The camera calibrating method according to the present invention thus 
constructed as previously mentioned can calibrate the second optical position information by 
using a portion of the automotive vehicle. 

[0034] In the camera calibrating apparatus according to the present invention, the imaging device 
is mounted on an automotive vehicle. The camera calibrating method according to the present 
invention thus constructed as previously mentioned can calibrate parameters of the optical 
section mounted on an automotive vehicle or the like to accurately detect the position of an 
object located on the road even if the housing unit is mounted on the inaccurate position of the 
automotive vehicle. 

[00351 In accordance with the present invention, there is provided a camera calibrating program 
of calibrating the position of an optical section supported by a housing unit of a camera for 
obtaining image information through the optical section, and allowing a computer to execute: a 
first housing position information storing step of storing first housing position information 
indicative of the position of the housing unit in the first coordination coordinate system; a second 
housing position information storing step of storing second housing position information 
indicative of the position of the housing unit in the second coordination coordinate system; a first 
optical position information producing step of producing first optical position information 
indicative of the position of the optical section in the fu'st coordination coordinate system on the 
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basis of the image information obtained by the imaging device in the first coordination 
coordinate system; a first optical position information storing step of storing the first optical 
position information produced in the first optical position information producing step; a second 
optical position information producing step of producing second optical position information 
indicative of the position of the optical section in the second coordination coordinate system 
fi-om the second housing position information stored in the second hoxxsing position information 
storing step on the basis of the first housing position information stored in the first housing 
position information storing step and the first optical position information stored in the first 
optical position information storing step; a second optical position information storing step of 
storing the second optical position information produced in the second optical position 
information producing step; and a calibrating step of calibrating the second optical position 
information stored in the second optical position information storing step on the basis of the 
image information obtained by the imaging device in the second coordination coordinate system. 
The camera calibrating program according to the present invention thus constructed as 
previously mentioned can calibrate parameters of the optical section mounted on an automotive 
vehicle or the like to accurately detect the position of an object located on the road. 
[0036] In accordance with the present invention, there is provided a camera calibrating program 
of calibrating the position of an optical section supported by a housing unit of a camera for 
obtaining image information through the optical section, and allowing a computer to execute: an 
optical position information storing step of storing optical position information indicative of the 
position of the optical section to a predetermined coordinate system; and a calibrating step of 
calibrating the optical position information stored in the optical position information storing step 
on the basis of the image information obtained by the imaging device on the predetermined 
coordinate system. The camera calibrating program according to the present invention thus 
constructed as previously mentioned can calibrate parameters of the optical section mounted on 
an automotive vehicle or the like to accurately detect the position of an object located on the 
road. 

[0037] In accordance with the present invention, there is provided a camera calibrating program 
of calibrating the position of an optical section supported by a housing unit of a camera for 
obtaining image information through the optical section, and allowing a computer to execute: a 
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first housing position information storing step of storing first housing position information 
indicative of the position of the housing unit in the first coordination coordinate system in which 
a revising marker is located; a second housing position information storing step of storing second 
housing position information indicative of the position of the housing unit in the second 
coordination coordinate system in which a calibrating marker is located; a fu-st optical position 
information producing step of producing first optical position information indicative of the 
position of the optical section in the first coordination coordinate system on the basis of the 
image information of the revising marker obtained by the imaging device; a first optical position 
information storing step of storing the first optical position information produced in the first 
optical position information producing step; a second optical position information producing step 
of producing second optical position information indicative of the position of the optical section 
in the second coordination coordinate system fi:om the second housing position information 
stored in the second housing position information storing step on the basis of the first housing 
position information stored in the first housing position information storing step and the furst 
optical position information stored in the first optical position information storing step; an 
estimated location information producing step of producing estimated location information 
indicative of the position of the calibrating marker to an image coordination coordinate system of 
the imaging device on the basis of the second optical position information produced in the 
second optical position information producing step; a second optical position information storing 
step of storing the second optical position information produced in the second optical position 
information producing step; an estimated location information storing step of storing the 
estimated location information produced in the estimated location information estimating step; 
and a calibrating step of calibrating the second optical position information stored in the second 
optical position information storing step on the basis of the image information of the calibrating 
marker obtained by the imaging device and the estimated location information stored in the 
estimated location information storing step. The camera calibrating program according to the 
present invention thus constructed as previously mentioned can calibrate the second optical 
position information by using the simple calibrating marker. 

[00381 In accordance with the present invention, there is provided a camera calibrating program 
of calibrating the position of an optical section supported by a housing unit of a camera for 
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obtaining image information through the optical section, and allowing a computer to execute: a 
first housing position information storing step of storing fu-st housing position information 
indicative of the position of the housing unit in the first coordination coordinate system in which 
a revising marker is located; a second housing position information storing step of storing second 
housing position information indicative of the position of the housing unit in the second 
coordination coordinate system; a first optical position information producing step of producing 
first optical position information indicative of the position of the optical section in the fu-st 
coordination coordinate systena on the basis of the image information obtained by the imaging 
device and indicative of the revising marker; a first optical position mformation storing step of 
storing the first optical position information produced in the first optical position information 
producing step; a second optical position information producing step of producing second optical 
position information indicative of the position of the optical section in the second coordination 
coordinate system fi:om the second housing position information stored in the second housing 
position information storing step on the basis of the first housing position information stored in 
the first housing position information storing step and the first optical position information stored 
in the first optical position information storing step; a second optical position information storing 
step of storing the second optical position information produced in the second optical position 
information producing step; and a calibrating step of calibrating the second optical position 
information stored in the second optical position information storing step on the basis of a 
motion vector of the image information obtained by the imaging device in the second 
coordination coordinate system. The camera calibrating program according to the present 
invention thus constructed as previously mentioned can calibrate the second optical position 
information by using the motion vector. 

[00391 In accordance with the present invention, there is provided a camera calibrating program 
of calibrating the position of an optical section supported by a housing unit of a camera for 
obtaining image information through the optical section, and allowing a computer to execute: a 
fu-st hoxising position information storing step of storing first housing position information 
indicative of the position of the housing unit in the first coordination coordinate system in which 
a revising marker is located; a second housing position information storing step of storing second 
housing position information indicative of the position of the housing unit in the second 
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coordination coordinate system in which an automotive vehicle is located; a first optical position 
information producing step of producing first optical position information indicative of the 
position of the optical section in the first coordination coordinate system on the basis of the 
image information of the revising marker obtained by the imaging device; a first optical position 
information storing step of storing the first optical position information produced in the first 
optical position information producing step; a second optical position information producing step 
of producing second optical position information indicative of the position of the optical section 
in the second coordination coordinate system firom the second housing position information 
stored in the second housing position information storing step on the basis of the first housing 
position information stored in the first housing position information storing step and the first 
optical position information stored in the first optical position information storing step; an 
estimated location information producing step of producing estimated location information 
indicative of the position of the automotive vehicle to an image coordination coordinate system 
of the imaging device on the basis of the second optical position information produced in the 
second optical position information producing step; a second optical position information storing 
step of storing the second optical position information produced in the second optical position 
information producing step; an estimated location information storing step of storing the 
estimated location information produced in the estimated location information estimating step; 
and a calibrating step of calibrating the second optical position information stored in the second 
optical position information storing step on the optical section basis of the image information of 
the automotive vehicle obtained by the imaging device and the estimated location information 
stored in the estimated location information storing step. The camera calibrating program 
according to the present invention thus constructed as previously mentioned can calibrate the 
second optical position information by using a portion of the automotive vehicle. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[00401 The features and advantages of a camera calibrating apparatus according to the present 
invention will be more clearly understood fi-om the following description taken in conjunction 
with the accompanying drawings: 
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r0041] FIG. 1 IS a block diagram showing a camera calibrating apparatus according to the first 
embodiment of the present invention and a camera exemplified as an imaging device; 
[0042] FIG. 2 is a perspective view showing the first coordination coordinate system in which 
the camera shown in FIG. 1 is located; 

r0043] FIG. 3 is a perspective view showing the second coordination coordinate system in which 
the camera shown in FIG. 1 is located; 

r0044] FIG. 4 is a perspective view showing a coordination coordiante system of the camera 
shown in FIG. 1; 

[0045] FIG. 5 is a plan view showing an image coordination coordinate system of the camera 
shown in FIG. 1 ; 

[0046] FIG. 6 is a perspective view showing a parallel translation of the camera shown in FIG. 1; 
[0047] FIG. 7 is a perspective view showing a rotational motion of the camera shown in FIG. 1; 
r0048] FIG. 8 is a side view showing a parallel translation of the camera shown in FIG. 1; 
r00491 FIG. 9 is a side view showing a rotational motion of the camera shown in FIG. 1; 
rOOSOl FIG. 10 is a block diagram showing an image location information extracting section of 
the camera calibrating apparatus shown in FIG. 1 ; 

[00511 FIG. 1 1 is a block diagram showing a computer and ECU to ensure the camera calibrating 
apparatus shown in FIG. 1 ; 

[0052] FIG. 12 is a block diagram showing a camera unit of the camera calibrating apparatus 
shown in FIG. 1; 

[0053] FIG. 13 is a flow chart showing a calibrating operation of the camera calibrating 
apparatus shown in FIG. 1; 

[0054] FIG. 14 is a block diagram showing a camera calibrating apparatus according to the 
second embodiment of the present invention and a camera exemplified as an imaging device; 
[0055] FIG. 15 a plan view showing an image coordination coordinate system of the camera 
shown in FIG. 14; 

[0056] FIG. 16 is a flow chart showing a calibrating operation of the camera calibrating 
apparatus shown in FIG. 14; 

[00571 FIG. 17 is a block diagram showing the camera calibrating apparatus according to the 
third embodiment of the present invention and a camera exemplified as an imaging device; 
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[0058] FIG. 18 is a perspective view showing the first coordination coordinate system in which 
the camera shown in FIG. 17 is located; 

[00591 FIG. 19 is a perspective view showing the second coordination coordinate system in 
which the camera shown in FIG. 17 is located; 

r00601 FIG. 20 is a perspective view showing a coordination coordinate system of the camera 
shown in FIG. 17; 

[0061] FIG. 21 is a side view showing an imaginary camera of the camera calibrating apparatus 
shown in FIG. 17; 

[0062] FIG. 22 is a schematic view showing an operation of the calibrating section of the camera 
calibrating apparatus shown in FIG. 17; 

r00631 FIG. 23 is a schematic view showing an operation of the calibrating section of the camera 
calibrating apparatus shown in FIG. 17; 

10064] FIG. 24 is a schematic view showing an operation of the calibrating section of the camera 
calibrating apparatus shown in FIG. 1 7; 

[00651 FIG. 25 is a schematic view showing an operation of the calibrating section of the camera 
calibrating apparatus shown in FIG. 1 7; 

r00661 FIG. 26 is a block diagram showing a computer and ECU to ensure the camera calibrating 
apparatus shown in FIG. 17; 

r0067] FIG. 27 is a block diagram showing a camera xmit of the camera calibrating apparatus 
shown in FIG. 17; 

[0068] FIG. 28 is a flow chart showmg a calibrating operation of the camera calibrating 

apparatus shown in FIG. 17; 

[0069] FIG. 29 is a block diagram showing a camera calibrating apparatus according to the 
fourth embodiment of the present invention and a camera exemplified as an image device; 
r0070] FIG. 30 is a perspective view showing the first coordination coordinate system in which 
the camera shown in FIG. 29 is located; 

[00711 FIG. 31 is a perspective view showing the second coordination coordinate system in 
which the camera shown in FIG. 29 is located; 

[00721 FIG. 32 is a perspective view showing a coordination coordinate system of the camera 
shown in FIG. 29; 
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[0073] FIG. 33 is a plan view showing an image coordination coordinate system of the camera 
shown in FIG, 29; 

[00741 FIG. 34 is a schematic view showing an operation of the calibrating section of the camera 
calibrating apparatus shown in FIG. 29; 

rOQ75I FIG. 35 is a schematic view showing an operation of the calibrating section of the camera 
calibrating apparatus shown in FIG. 29; 

r00761 FIG. 36 is a block diagram showing a computer and ECU to ensure the camera calibrating 
apparatus shown in FIG. 29; 

[00771 FIG. 37 is a block diagram showing a camera unit of the camera calibrating apparatus 
shown in FIG. 29; 

[00781 FIG. 38 is a flow chart showing a calibrating operation of the . camera calibrating 
apparatus shown in FIG. 29; 

[00791 FIG. 39 is a block diagram of a conventional camera calibrating apparatus and a camera 
exemplified as an imaging device; 

[00801 FIG. 40 is a perspective view showing the first coordination coordinate system in which 
the camera shown in FIG. 39 is located; and 

[00811 FIG. 41 is a perspective view showing the second coordination coordinate system in 
which the camera shown in FIG. 39 is located. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[00821 The embodiments of the camera calibrating apparatus according to the present invention 
will now be described in detail in accordance with accompanying drawings. 
[00831 (First Embodhnent) 

[00841 Referring now to the drawings, in particular to FIGS. 1 to 13, there is shown a first 
embodhnent of a camera calibrating apparatus accordmg to the present mvention. 
[00851 The construction of the first embodiment of the camera calibratmg apparatus according to 
the present invention will now be described hereinafter. As shown m FIGS. 1 to 3, the camera 
calibrating apparatus 100 is electrically connected to a camera 110 exemplified as an imaging 
device. The camera 110 includes a housing unit 1 1 1 and an optical section 112 supported by the 
housing unit 111 to obtain image information through the optical section 112. 
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[00861 Th^ camera calibrating apparatus 100 comprises a first housing position information 
storing section 1 15 for storing first housing position information indicative of the position of the 
housing unit 111 in the first coordination coordinate system 101, a second housing position 
information storing section 116 for storing second housing position information indicative of the 
position of the housing unit 111 in the second coordination coordinate system 102, a revising 
marker position information storing section 125 for storing revising marker position information 
indicative of the position of a revising marker 105 in the first coordination coordinate system 
101, and a calibrating marker position information storing section 126 for storing calibrating 
marker position information indicative of the position of a calibrating marker 106 in the second 
coordination coordinate system 1 02. 

[00871 The first coordination coordinate system 101 is defined in a first working space such as a 
camera production plant to have Xi-axis, Yi-axis, and Zi-axis, while the revising marker 105 laid 
out in the first working space causes the camera calibrating apparatus 100 to revise the camera 
1 10 located in the first working space. The revising marker 105 is constituted by a plurality of 
marking objects which are respectively located in predetermined positions on the first 
coordination coordinate system 101, and which are within a viewing field of the camera 110 
located in the first working space. 

[00881 The second coordination coordinate system 102 is defined in a second working space 
such as an automotive vehicle production plant to have X2-axis, Y2-axis, and Z2-axis, while X2- 
airis and Y2-axis collectively defme X2-Y2 plane which is represented by a road surface 102a on 
which an automotive vehicle 108 is located. The calibrating marker 106 is constituted by two or 
more marking objects which are respectively located at predetermined positions of the road 
surface 102a in the second coordination coordinate system 102. 

[00891 The camera calibrating apparatus 100 is adapted to revise the camera 110 in the first 
working space. The camera 110 is located at a predetermined position on the fu^st coordination 
coordinate system 101, while the first housing position information storing section 115 is 
adapted to store first housing position information indicative of the position of the housing unit 
111. Here, the term "the calibration of the camera 1 10" is intended to indicate the calculation of 
the accurate position of the optical section 112 of the camera 110 mounted on the automotive 
vehicle 108. 
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r00901 The camera 110 revised by the camera calibrating apparatus [[110]] 100 is mounted on 
the automotive vehicle 108 to be located at a predeteraiined position on the second coordination 
coordinate system 102 in the second working space, while the second housing position 
information storing section 116 is adapted to store second housing position information 
indicative of the position of the housing unit 111. Here, the second housing position information 
indicates an accurate position in which the housing unit 111 of the camera 1 10 is located when 
the camera 1 10 is mounted on the automotive vehicle 108. 

r00911 The camera calibrating apparatus 100 further comprises a first optical position 
information producing section 117 for producing furst optical position information indicative of 
the position of the optical section 1 12 in the first coordination coordinate system 101, and a first 
optical position information storing section 118 for storing the first optical position information 
produced by the furst optical position information producing section 117. 
r00921 The first optical position information producing section 117 is adapted to calculate the 
position of the optical section 112 in the first coordination coordinate system 101 fi-om the 
calibrating marker position information stored in the calibrating marker position information 
storing section 125 on the basis of the image information of the calibrating marker 105 obtained 
by the camera 110. Here, the term "the position of the optical section 1 12" is intended to indicate 
an optical center of the optical section 112 and the position of an optical axis, and other optical 
coordinate parameters. The calculation of the position of the optical section 112 in the first 
coordination coordinate system 101 is performed on the basis of a method disclosed in the 
document 1 (R. Tsai, A versatile camera calibration technique for high-accuracy 3D machine 
vision metrology using off-the-shelf TV cameras and lenses, IEEE Journal of Robotics and 
Automation, RA-3 (4): 323-344, 1987). 

[00931 The camera calibrating apparatus 100 further comprises a second optical position 
information producing section 120 for producing second optical information indicative of the 
position of the optical section 112 in the second coordination coordinate system 102, and a 
second optical position information storing section 130 for storing the second optical position 
information produced by the second optical position information producing section 120. 
[0094] The second optical position information producing section 120 is adapted to produce 
second optical position information indicative of the position of the optical section 112 in the 
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second coordination coordinate system 102 from the second housing position information stored 
in the second housing position information storing section 1 16 on the basis of the first housing 
position information stored by the first housing position information storing section 115 and the 
first optical position information stored in the first optical position information storing section 
118. 

r00951 The second optical position information producing section 120 is adapted to calculate the 
position of the optical section 1 12 in the second coordination coordinate system 102 on the basis 
of a following method. 

[00961 The method is of comparing the position of the housing unit 1 1 1 to the first coordination 
coordinate system 101 with the position of the optical section 112 in the first coordination 
system 101 to obtain positional information between the housing unit 111 and the optical section 
112, and to calculate information indicative of the position of the optical section 112 in the 
second coordination coordinate system 102 from the position of the housmg unit 111 in the 
second coordination coordmate system 102 on the basis of the positional information between 
the housing unit 111 and the optical section 112. Consequentiy, the second optical position 
information indicates the position of the optical section 1 12 imder the condition that the housing 
unit 1 11 is located at an accurate position of the automotive vehicle 108. 
r0097] As shown in FIG. 4, the camera 110 mounted on the automotive vehicle 108 in the 
second working space defines a camera coordination coordinate system 113 having x-axis, y- 
axis, and z-axis on the basis of the second optical position information. The origin point of the 
camera coordination coordinate system 113 is in register with the optical center of the optical 
section 112. The x-axis of the camera coordination coordinate system 113 extends in a horizontal 
direction of the camera 110. The y-axis of the camera coordination coordinate system 113 
extends in a vertical direction of the camera 110. The z-axis of the camera coordination 
coordinate system 1 13 is in axial alignment with the optical axis of the optical section 112. 
r00981 The image coordination coordinate system 114 is defmed on a plane spaced apart from 
the origin point of the camera coordination coordinate system 113 with a focal length "f in the 
direction of the z-axis of the camera coordination coordinate system 113. The image 
coordination system 114 has p-axis and q-axis. The camera 110 is adapted to obtain, as image 
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information, an optical image focused on the image coordination coordinate system 114 through 
the optical section 112. 

r0099] The camera calibrating apparatus 100 further comprises an estimated location position 
information producing section 140 for producing estimated location position information of the 
calibrating marker 106 to the image coordination coordinate system 1 14 of the camera 110, and 
an estimated location position mformation storing section 150 for storing the estimated location 
position information produced by the estimated location position information producing section 
140. 

[01001 The estimated location position information producing section 140 is adapted to produce 
estimated location position information of the calibrating marker 106 to the image coordination 
coordinate system 1 14 of the camera 110 from the position information of the calibrating marker 
106 stored in the calibrating marker position information storing section 126 on the basis of the 
second optical position information produced by the second optical position information 
producing section 120. The calculation of the estimated location of the calibrating marker 106 to 
the image coordination coordinate system 1 14 of the camera 1 10 is performed on the basis of the 
above mentioned document 1 . 

[0101] As shown in FIG. 5, the calibrating marker 106 located in the second coordination 
coordinate system 102 is focused on the image location Pn' (n =1, 2, 3, 4, 5, 6) of the image 
coordination coordinate system 114 through the optical section 112. Here, the image location Pn' 
is in register with the estimated image location Pn (n=l, 2, 3, 4, 5, 6) calculated by the estimated 
image location calculating section 140 xmder the condition that the housing imit 1 1 1 is located at 
an accurate position of the automotive vehicle 108, more specifically, the housing unit 111 is 
located at the position represented by the second housing position information with no deviation 
to the accurate position. However, the housing unit 111, in general, is located at an inaccurate 
position with deviation to the accurate position. This leads, to the fact that the second optical 
position information is produced with deviation to the accurate position. As a result, the 
estimated location Pn is spaced apart from the image location Pn* of the image coordination 
coordinate system 114. 

[0102] The second optical position information includes six parameters indicative of a parallel 
displacement and a rotational motion of the camera coordination coordinate system 1 13 to the 
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second coordination coordinate system 102. The six parameters are constituted by the parallel 
displacement components of the X2-axis, the Y2-axis, and the Z2-axis shown in FIG. 6 and the 
rotational motion of the x-axis, the y-axis, and the z-axis shown m FIG. 7. The deviation of the 
second optical position information includes components of the parallel displacement and 
rotational motion. The housing unit 111 is, in general, located on the automotive vehicle with the 
parallel displacement which is within the range of a few centimeters, and the rotational motion 
which is within the range of a few degrees. 

r01031 The following description will be directed to the case tiiat the image information of the 
road surface 102a obtained by the camera 1 10 is displayed in conjunction with an additional line 
when the automotive vehicle 108 is operated by an operator. Here, the housing unit 111 is 
located at the position of the automotive vehicle 108 which is spaced apart from the road surface 
302a, while the additional line is located at the position spaced apart from a rear portion of the 
automotive vehicle 108. The height of the housing unit 1 1 1 to the road surface 302a is equal to 
1,000 millimeters, while the distance of the additional line to the rear portion is equal to 3000 
millimeters. 

[01041 When the housing unit 1 1 1 is located at the position spaced apart from a design position 
of the automotive vehicle 108 in the Y-axis direction, more specifically, the parallel shift 
displacement of the housing unit 111 to the design position is equal to 50 millimeters as shown 
in FIG. 8, the displacement of the additional line is equal to 50 millimeters in Yi-axis direction. 
In this case, the elongation value of the image location Pn' to the estimated location Pn on the 
image coordination coordinate system 114 shown in FIG. 5 can be negligible. 
rOlOSI When, on the other hand, the housing unit 1 1 1 is mounted on the automotive vehicle 108 
with a rotational displacement of the housing unit 1 1 1 to the X2-axis direction, more specifically, 
the rotational displacement of the housing unit 1 1 1 to the X2-axis direction is equal to one degree 
as shown in FIG. 9, the additional line is located with a displacement of 184 millimeters in the 
Y2-axis direction. In this case, the elongation value of the image location Pn* to the estimated 
location Pn on the image coordination coordinate system 114 shown in FIG. 5 cannot be 
negligible. 

[01 06") As will be seen from the above description, the second optical position information may 
be calibrated in consideration with only the rotational displacement of the optical section 112. 
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r01071 The camera calibrating apparatus 100 further comprises a calibrating section 160 for 
calibrating the second optical position information stored in the second optical position 
information storing section 130 to order to calibrate the above mentioned deviation of the second 
optical position information. 

rOlOS] The calibrating section 1 60 is adapted to calibrate the second optical position information 
stored in the second optical position information storing section 130 on the basis of the image 
information of the calibrating marker 106 obtained by the camera 1 10 and the estimated location 
position information stored in the estimated location position information storing section 1 50. 
r01091 The calibrating section 160 includes an image location information extracting section 170 
for extracting image location information indicative of an image location of the calibrating 
marker 106 to the image coordination coordinate system 1 14 of the camera 1 10 on the basis of 
the image information of the calibrating marker 106 obtained by the camera 110, a calibration 
value calculating section 180 for calculating a calibration value of the second optical position 
information stored in the second optical position information storing section 130 on the basis of 
the image location information extracted by the image location information extracting section 
170 and the estimated location position information stored in the estimated location position 
information storing section 150, and an optical position information calibrating section 190 for 
calibrating the second optical position information stored in the second optical position 
information storing section 130 on the basis of the calibration value calculated by the calibration 
value calculating section 180. 

rOllO] The image location information extracting section 170 includes an image displaying 
section 171 for displaying an image of the calibrating marker 106 obtained by the camera 110, 
and an image location specifying section 172 for specifying the image location Pn' of the 
calibrating marker 106 in the image of the calibrating marker 106 displayed by the image 
displaying means 171 in order to extract the image location information from the image 
information. 

roil 11 As shown m FIG. 10, the image displaying section 171 is adapted to display both the 
image of the calibrating marker 106 obtained by the camera 110 and a cursor 174 for specifying 
the image location Pn* of the calibrating marker 106. 
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[0112] The image location specifying section 172 includes an up-arrow key 175a, a down-arrow 
key 175b, a left-arrow key 175c, and a right-arrow key 175d, all of which are capable of 
allowing the cursor 174 displayed on the image mformation displaying section 171 to be moved 
in the respective directions, an enter key 176 for deciding the position of the cursor 174 as the 
image location Pn* of the calibrating marker 106, and a number displaying section 177 for 
displaying number "n" of the marking object of the calibrating marker 106, and an increment key 
178a and a decrement key 178b each for changing the number "n" displayed on the number 
displaying section 177. 

[0113] The calibration value calculating section 180 is adapted to calculate the calibration value 
of the second optical position information on the basis of the following method. 
[01 141 The following direction will be directed to the case that the deviation of the rotational 
component of the second optical position information is considered as an overall deviation of the 
second optical position information. 



N 
n'=l 



{pn {Ri3 pn + R^^qn + i?33/) - f{Rup + Ri\q + j?31 / )f 

+ (^n'(i?13p + i?23^« + i?33/)- f(R\lp + R22q+Rzif)f_ 



(1) 



[0115] (pn'j qn') and (pn, qn) are linked by the following formula (2) on the basis of the formula 
(1). 



Rup + R2\q + Ri] f 



R\2p + Rnq + R%if 

, ^Rnp + R22q + R32f 
q=f ^ 2 ±. 

Rnp + R23q + R33f 



(2) 



[0116] Rii to R33 and 0, 0, 9 are linked by the following formulas (3), (4), and (5). 
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Rx= 0 cosd sind 
0 -sin^ cos^ 
cos ^ 0 - sin ^ 
Ry=^ 0 1 0 
sin{z5 0 cos(z) 

cos (p sin ^ 0 
i?2= -sin^ cos^ 0 
0 0 1 



(3) 



R^RxxRyxHz (4) 



Rn Rn Rn 
R= Rii Rn R23 

i?31 R32 R33 



(5) 



roil 71 In this embodiment, the calibrating marker 106 is constituted by six marking objects, 
however, the calibrating marker 106 may be constituted by two or more marking objects. On the 
other hand, the calibration value calculating section 180 can calculate 9, O, and (p if the 
calibrating marker 106 is constituted by two marking objects. Additionally, the calibration value 
calculating section 180 can calculate 9, O, and (p at a relatively high accuracy if the calibrating 
marker 106 is constituted by three or more marking objects. 

roil 81 As shown in FIG. 1 1, the camera calibrating apparatus 100 thus constructed as previously 
mentioned can be realized by comprising the computer 191 for calibrating the camera 110, and 
th e ECU 192 an electronic control unit (hereinafter simply referred to as "ECU") 192 constituted 
as an image controlling apparatus for controlling the camera 110. 

r01191 The computer 191 is shown in FIG. 11(a) as including CPU, Ri\M, ROM a central 
processing unit (hereinafter simply referred to as ^*CPU"\ a random access memory (hereinafter 
simply referred to as "RAM"), a read only memory (hereinafter simply referred to as "ROM"'), 
an input/output interface, and other elements. The computer 191 is electrically connected to the 
camera 1 10 at the first working space. In this embodiment, the computer 191 constitutes each of 
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the first housing position information storing section 115, the second housing position 
information storing section 116, the first optical position information producing section 1 17, the 
first optical position information storing section 118, the second optical position information 
producing section 120, the calibrating marker position information storing section 125, the 
calibrating marker position information storing section 126, the second optical position storing 
section 130, the estimated location position information producing section 140, and the estimated 
location position information storing section 150. 

r01201 The ECU 192 is shown in FIG. 11(c) as mcluding CPU, RAM, ROM, an input/output 
interface, and other elements. The ECU192 is mounted on the automotive vehicle 108 to be 
electrically connected to the camera 110 in the second working space. In this embodiment, the 
ECU 192 constitutes each of the second optical position information storing section 130, the 
estimated location position information storing section 150, and the calibrating section 160. 
r012n As will be seen fi-om FIG. 11(b), the camera 110 is transferred to the second working 
space fi-om the first working space with a recordable medium 193 such as for example CD ROM 
compact disk read only memory (hereinafter simply referred to as "CD-ROM" and a magnetic 
disk. The recordable medium 193 has stored therein each of the second optical position 
information and the estimated location position information to ensure that the each of the second 
optical position information and the estimated location position information is effectively 
transferred to the ECU 192 fi-om the computer 191 . 

r01221 In this embodiment, the camera 1 10 and the recordable medium 193 are transferred to the 
second working space from the first working space, however, the camera 1 10, the second optical 
position information storing section 130, the estimated location position information storing 
section 150, and the calibrating section 160 may be collectively transferred as a camera assembly 
194 to the second working space from the first working space as will be seen from FIG. 12. 
[0123] The following description will be directed to the operation of the first embodiment of the 
camera calibrating apparatus according to the present invention. The second optical position 
information is calibrated by the camera calibrating apparatuis 100 through the steps shown in the 
FIG. 13. 

[0124] The camera 1 10 is firstly located at the predetermined position on the first coordination 
coordinate system 101 in the first working space (in the step SlOl). The first housing position 
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information, the second housing position information, the revising marker position information, 
and the calibrating marker position information are then stored in the first housing position 
information storing section 115, the second housing position information storing section 1 16, the 
revising marker position information storing section 125, and the calibrating marker position 
information storing section 126 (in the step S 102), respectively. Here, the first housing position 
information, the second housing position information, the revising marker position information, 
and the calibrating marker position information are obtained on the basis of positions 
predetermined in the design process and measured by a measuring instrument. 
[01251 The image information of the revising marker 105 is then obtained by the camera 1 10 (in 
the step SI 03). The first optical position information is then produced by the first optical position 
information producing section 117 fi*om the revising marker position information stored in the 
revising marker position information storing section 125 on the basis of the image information of 
tiie revismg marker 105 obtamed by the camera 1 10 (in the step SI 04). The first optical position 
information produced by the fu^t optical position information producing section 117 is then 
stored in the fu:st optical position information storing section 1 18 (in the step SI 05). 
[0126] The second optical position information is then produced by the second optical position 
information producing section 120 fi-om the second housing position information stored in the 
second housing position information storing section 1 16 on the basis of the first housing position 
information stored in the furst housing position information storing section 115 and the first 
optical position information stored in the first optical position information storing section 1 18 (in 
the step SI 06). The second optical position information produced by the second optical position 
information producing section 120 is then stored in the second optical position information 
storing section 130 (in the step SI 07). 

[0127] The estimated location position information indicative of the estimated location of the 
calibrating marker 106 is then produced by the estimated location position information producing 
section 140 from the calibrating marker position information stored in the calibrating marker 
position information storing section 126 on the basis of the second optical position information 
produced by the second optical position mformation producing section 120 (in the step SI 08). 
The estimated location position information produced by the estimated location position 
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information producing section 140 is stored in the estimated location position information storing 
section 1 50 (in the step S 1 09). 

[01281 The camera 110 and the recordable medium 193 are then transferred to the second 
working space from the first working space. The camera 110 is then mounted to the automotive 
vehicle 108 to be located at the predetermined position on the second coordination coordinate 
system 102 (in the step SI 10). 

r01291 The image information of the calibrating marker 106 is then obtained by the camera 110 
(in the step Sill). The image information of the calibrating marker 106 obtained by the camera 
110 is then displayed on the image displaying section 171 (in the step SI 12) as shovm in FIG. 
10. The image location Pn* of the calibrating marker is specified by the image location specifying 
section 172 (in the step SI 13). Here, the increment key 178a and the decrement key 178b are 
operated by an operator, while the number "n" of the marking object of the calibrating marker 
106 displayed by the number displaying section 177 in response to the increment key 178a and 
the decrement key 178b. The up-arrow key 175a, the down-arrow key 175b, the lefl-arrow key 
175c, and the right-arrow key 175d are operated by the operator, while the cursor 174 displayed 
on the image displaying section 171 is moved in response to each of the up-arrow key 175a, the 
down-arrow key 175b, the left-arrow key 175c, and the right-arrow key 175d. The enter key 176 
is operated by the operator, while the position of the cursor 176 is decided as the image location 
Pn' of the number "n" of the calibrating marker 106 displayed on the number displaying section 
177. 

[01301 The calibration value of the second optical position information stored in the second 
optical position information storing section 130 is then calculated by the calibration value 
calculating section 1 80 on the basis of the image location information extracted by the image 
location information extracting section 170 and the estimated location position information 
stored in the estimated location position information storing section 150 (in the step SI 14). 
f01311 The second optical position information stored in the second optical position information 
storing section 130 is finally calibrated by the optical position information calibrating section 
190 on the basis of the calibration value calculated by the calibration value calculating section 
180 (in the step SI 15). In this embodiment, the above mentioned steps 101 to 115 are executed 
as a program by a computer. 
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rQ1321 From the above detail description, it will be understood that the present embodiment of 
the camera calibrating apparatus 100 according to the present invention can calibrate the optical 
parameters of the camera 110 mounted on the automotive vehicle 108 to ensure an accurate 
position of an object located on the road is detected. 

101331 In this embodiment, the camera calibrating apparatus 100 according to the present 
invention can calibrate the second optical position by using the simple calibrating marker 106. 
r01341 In this embodiment, the camera calibrating apparatus 100 according to the present 
invention can specify the unage location of the calibrating marker 106 to ensure that the image 
location information indicative of the image location of the calibrating marker 106 is extracted. 
[0135] (Second Embodiment) 

r01361 Referring now to the drawings, in particular to FIGS. 14 to 16, there is shown a second 
embodiment of a camera calibrating apparatus accordmg to the present invention. 
[0137] The construction of the second embodiment of the camera calibrating apparatus according 
to the present invention will now be described hereinafter. The constitutional elements of the 
second embodiment of the camera calibrating apparatus the same as those of the first 
embodiment of the camera calibrating apparatus vnW not be described but bear the same 
reference numerals and legends as those of the first embodiment of the camera calibrating 
apparatus. 

r01381 In FIG. 14, the camera calibrating apparatus 200 comprises a calibrating section 260 for 
calibrating the second optical position information stored in the second optical position 
information storing section 130. 

[01391 The calibrating section 260 is adapted to calibrate the second optical position information 
stored in the second optical position information storing section 130 on the basis of the image 
information of the calibrating marker 106 obtained by the camera 110 and the estimated location 
position information stored in the estimated location position information storing section 1 50. 
[01401 The calibrating section 260 includes an image location information extracting section 270 
for extracting image location information indicative of an image location of the calibrating 
marker 106 to the image coordination coordinate system 114 of the camera 110 on the basis of 
the image information of the calibrating marker 106 obtained by the camera 110, a calibration 
value calculating section 280 for calculating a calibration value to the second optical position 
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information stored in the second optical position information storing section 130 on the basis of 
the image location information extracted by the image location information extracting section 
270 and the estimated location position information stored in the estimated location position 
information storing section 150, and an optical position information calibrating section 290 for 
calibrating the second optical position information stored in the second optical position 
information storing section 130 on the basis of the calibration value calculated by the calibration 
value calculating section 280. 

[0141] The image location information extracting section 270 includes an estimated area 
information storing section 271 for storing estimated area information (see FIG. 15) indicative of 
an estimated area 274 of the calibrating marker 106 to the image coordination coordinate system 
114 of the camera 110, and an image location searching section 272 for searching the image 
location Pn' of the calibrating marker 106 from the image information of the calibrating marker 
106 obtained by the camera 110 on the basis of the estimated area information stored by the 
estimated area information storing section 271 and the estimated location position information 
stored by the estimated location position information storing section 150 to ensure that the image 
location information is extracted. 

r0142] The calibration value calculating section 280 is adapted to calculate the calibration value 
to the second optical position information on the basis of a method which is the same as the 
method of the calibration value calculating section 180 of the first embodiment. 
r01431 The following description will be directed to the operation of the second embodiment of 
the camera calibrating apparatus according to the present invention. The second optical position 
information is calibrated by the camera calibrating apparatus 200 through the following steps 
shown in FIG. 16. 

[0144] The camera 1 10 is firstly located at the predetermined position on the first coordination 
coordinate system 101 in a first working space (in the step 8201). The fu:st housing position 
information, the second housing position information, the revising marker position information, 
and the calibrating marker position information are then stored in the first housing position 
information storing section 115, the second housing position information storing section 1 16, the 
revising marker position information storing section 125, and the calibrating marker position 
information storing section 126 (in the step S202), respectively. Here, the first housing position 
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infonnation, the second housing position information, the revising marker position information, 
and the calibrating marker position information are obtained on the basis of positions 
predetermined in the design process and measured by a measuring instrument. 
r01451 The image information of the revising marker 105 is then obtained by the camera 1 10 (m 
the step 8203). The furst optical position information is then produced by the first optical position 
mformation producing section 117 from the revismg marker position information stored in the 
revising marker position information storing section 125 on the basis of the image information of 
the revising marker 105 obtained by the camera 1 10 (m the step S204). The first optical position 
information produced by the first optical position information producing section 117 is then 
stored in the first optical position information storing section 1 1 8 (in the step S205). 
[01461 The second optical position information is then produced by the second optical position 
information producing section 120 from the second housmg position information stored in the 
second housing position information storing section 1 16 on the basis of the first housing position 
information stored in the first housing position information storing section 115 and the first 
optical position information stored in the first optical position information storing section 1 18 (in 
the step S206). The second optical position information produced by the second optical position 
information producing section 120 is then stored in the second optical position information 
storing section 130 (in the step S207). 

[0147] The estimated location position information indicative of the position of the calibratmg 
marker 106 to the image coordination coordinate system of the camera 1 10 is then produced by 
the estimated location position information producing section 140 from the calibrating marker 
position information stored in the calibrating marker position information storing section 126 on 
the basis of the second optical position information produced by the second optical position 
information producing section 120 (in the step S208). The estimated location position 
information produced by the estimated location position information producing section 140 is 
stored in the estimated location position information storing section 150 (in the step S209). 
r01481 The camera 110 and the recordable medium 193 are then transferred to the second 
working space from the first working space. The camera 1 10 is then mounted to the automotive 
vehicle 108 to be located at the predetermined position on the second coordination coordinate 
system 102 (in the step 8210). 
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r01491 The image information of the calibrating marker 106 is then obtained by the camera 1 10 
(in the step 82 1 1). As shown in FIG. 15, the image location Pn' is searched by the image location 
searching section 272 in the estimated area 274 at the estimated location Pn of the calibrating 
marker 106. The image location information is then extracted by the image location information 
extracting section 270. 

[0150] The calibration value to the second optical position information stored in the second 
optical position information storing section 130 is then calculated by the calibration value 
calculating section 280 on the basis of the image location information extracted by the image 
location information extracting section 270 and the estimated location position information 
stored in the estimated location position information storing section 150 (in the step S213). 
["0151] The second optical position information stored in the second optical position information 
storing section 130 is finally calibrated by the optical position information calibrating section 
290 on the basis of the calibration value calculated by the calibration value calculating section 
280 (in the step 82 14). In this embodiment, the above mentioned steps 101 to 115 may be 
executed as a program by a computer. 

|"0152] From the above detailed description, it will be understood that the second embodiment of 
the camera calibrating apparatus 200 can easily extract the image location information of the 
calibrating marker 106 by reason that the image location of the calibrating marker 106 is 
searched by the image location searching section 272. 
10153] (Third Embodiment) 

[0154] Referring now to the drawings, in particular to FIGS. 17 to 28, there is shown a third 
embodiment of a camera calibrating apparatus according to the present invention. 
[0155] The construction of the third embodiment of the camera calibrating apparatus according 
to the present invention will now be described hereinafter. 

[0156] As shown in FIGS. 17 to 19, the camera calibrating apparatus 300 is electrically 
connected to a camera 310 as an imaging device. The camera 310 mcludes a housing unit 311 
and an optical section 312 supported by the housing unit 311 to obtain image information 
through the optical section 312. 

[0157] The camera calibrating apparatus 300 comprises a first housing position information 
storing section 315 for storing first housing position information indicative of the position of the 
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housing unit 311 to the first coordination coordinate system 301, a second housing position 
information storing section 316 for storing second housing position information indicative of the 
position of the housing unit 3 1 1 in the second coordination coordinate system 302, and a revising 
marker position information storing section 325 for storing revising marker position information 
indicative of the position of a revising marker 305 in the first coordination coordinate system 
301. 

[0158] The first coordination coordinate system 301 is defined in a fij-st working space such as a 
camera production plant to have Xi-axis, Yi-axis, and Zi-axis, while the revising marker 305 laid 
out in the first working space causes the camera calibrating apparatus 300 to revise the camera 
310 located in the first working space. The revising marker 305 is constituted by a plurality of 
marking objects which are respectively located at predetermined positions on the first 
coordination coordinate system 301, and which are within a viewing field of the camera 310 
located in the first working space. 

[01591 The second coordination coordinate system 302 is defined in a second working space 
such as an automotive vehicle production plant. The second coordination coordinate system 302 
has X2-axis, Y2"axis, and Z2-axis. The X2-axis and the Y2-axis collectively define X2-Y2 plane to 
be represented by a road surface 302a on which an automotive vehicle 308 is located. The 
dividing marker 307 is located on a bumper 309 of the automotive vehicle 308. The dividing 
marker 307 is constituted by two marking objects which is within a viewing field of the camera 
310 mounted on the automotive vehicle 308. The dividing marker 307 is located directiy below 
the camera 310 mounted on the automotive vehicle 308. 

[01601 The camera calibrating apparatus 300 is adapted to revise the camera 310 in the first 
working space. The camera 310 is located at a predetermined position on the first coordination 
coordinate system 301, while the first housing position information storing section 315 is 
adapted to store first housing position information indicative of the position of the housing unit 
311. Here, the term "the calibration of the camera 310" is intended to indicate the calculation of 
tiie accurate position of the optical section 312 of the camera 310 mounted on the automotive 
vehicle 308. 

[01611 The camera 310 revised by the camera calibrating apparatus 310 is mounted on the 
automotive vehicle 308 to be located at a predetermined position on the second coordination 
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coordinate system 302 in the second working space, while the second housing position 
information storing section 316 is adapted to store second housing position information 
indicative of the position of the housing unit 311. Here, the second housing position information 
indicates an accurate position in which the housing unit 311 of the camera 310 is located when 
the camera 3 1 0 is mounted on the automotive vehicle 308. 

r01621 The dividing marker 307 is located on the bumper 309 of the automotive vehicle 308 in 
spaced relationship with the position of the housing unit 311 represented by the second housing 
position information. This means that the dividing marker 307 is not moved with respect to the 
housing imit 311 when the automotive vehicle 308 is traveling on the road 302a. 
[0163] The camera calibrating apparatus 300 further comprises a first optical position 
information producing section 317 for producing first optical position information indicative of 
the position of the optical section 312 to the first coordination coordinate system 301, and a furst 
optical position information storing section 318 for storing the first optical position mformation 
produced by the first optical position information producing section 317. 
r0164] The first optical position information producing section 317 is adapted to calculate the 
position of the optical section 312 to the first coordination coordinate system 301 from the 
revising marker position information stored in the revising marker position information storing 
section 325 on the basis of the image information of the revising marker 305 obtained by the 
camera 310. Here, the term "the position of the optical section 312" is intended to indicate an 
optical center of the optical section 312 and the position of an . optical axis, and other optical 
parameters. The calculation of the position of die optical section 312 to the first ooordmation 
coordinate system 301 is performed on the basis of the method disclosed in the docmnent 1. 
[0165] The camera calibrating apparatus 300 further comprises a second optical position 
information producing section 320 for producing second optical information indicative of the 
position of the optical section 312 in the second coordination coordinate system 302, and a 
second optical position information storing section 330 for storing the second optical position 
information produced by the second optical position information producing section 320. 
[0166] The second optical position information producing section 320 is adapted to produce 
second optical position information indicative of the position of the optical section 312 in the 
second coordination coordinate system 302 fi-om the second housing position information stored 
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in the second housing position information storing section 316 on the basis of the first housing 
position information stored by the furst housing position information storing section 315 and the 
first optical position information stored in the first optical position information storing section 
318. 

[0167] The second optical position information producing section 320 is adapted to calculate the 
position of the optical section 3 12 in the second coordination coordinate system 302 on the basis 
of a method which is the same as the method of the second optical position information 
producing section 120 of the first enibodiment. 

[0168] As shown in FIG. 20, the camera 310 mounted on the automotive vehicle 308 located in 
the second working space defines a camera coordination coordinate system 313 having x-axis, y- 
axis, and z-axis on the basis of the second optical position information. The origin point of the 
camera coordination coordinate system 313 is in register with the optical center of the optical 
section 312. The x-axis of the camera coordination coordinate system 313 extends in a horizontal 
direction of the camera 310. The y-axis of the camera coordination coordinate system 313 
extends in a vertical direction of the camera 310. The z-axis of the camera coordination 
coordinate system 3 13 is in axial alignment with the optical axis of the optical section 312. 
[0169] The image coordination coordinate system 314 is defined on a plane spaced apart fi-om 
the origin point of the camera coordination coordinate system 313 with a focal length "f ' in the 
direction of the z-axis of the camera coordination coordinate system 313. The image 
coordination system 314 has p-axis and q-axis. The camera 310 is adapted to obtain, as image 
information, an optical image focused on the image coordination coordinate system 314 through 
the optical section 312. 

[0170] The camera calibrating apparatus 300 further comprises a calibrating section 360 for 
calibrating the second optical position information stored in the second optical position 
information storing section 330 on the basis of the motion vector of the image information 
obtained by the camera 3 10 in the second coordination coordinate system 302. 
[0171] The calibrating section 360 includes a plane-projected image producing section 361 for 
producing a plane-projected image fi-om the image information obtained by the camera 310 on 
the second coordination coordinate system 302, a plane-projected image dividing section 362 for 
dividing the plane-projected image produced by the plane-projected image producing section 361 
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into a plurality of image segments, a motion vector extracting section 363 for extracting the 
motion vector from the image segments divided by the plane-projected image dividing 
section362, a calibration value calculating section 364 for calculating a calibration value of the 
second optical position information stored by the second optical position information storing 
section 330 on the basis of the motion vector extracted by the motion vector extracting section 
363, and an optical position information calibrating section 365 for calibrating the second optical 
position information stored by the second optical position information storing section 330 on the 
basis of the calibration value calculated by the calibration value calculating section 364. 
[0172] As shown in FIG. 21, the plane-projected image produced by the plane-projected image 
producing section 361 is the sama as an image taken by an imaginary camera 370 located on a 
plane which defines an image coordination coordinate system 371 parallel to the road surface 
302a. The image of the road surface 302a focused on the image coordination coordinate system 
371 is the same as the reduced- size road surface 302a. 

[0173] The plane-projected image producing section 361 is adapted to produce plane-projected 
image on the basis of a following method. 

[0174] In FIG. 20, the angle of the x-axis of the image coordination coordinate system 313 to the 
road surface 302a is represented by a legend a. The angle of the y-axis of the image coordination . 
coordinate system 313 to the road surface 302a is represented by a legend p. The distance 
between the origin point of the camera coordination coordinate system 313 and an intersecting 
point of the extension line of the z-axis of the camera coordination coordinate system 313 to the 
road surface 302a is represented by a legend "c". The X2-Y2 surface of the second coordination 
coordinate system 302, i.e., the road surface 302a is represented by following equations (6) and 



(7). 




(6) 



(7) 
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r01751 The transformation to the camera coordination coordinate system 313 from the image 
coordination coordinate system 314 is represented by a following equation (8). Here, the legend 
(p, q) represents ordinate and abscissa of the image location P* to the im^e coordination 
coordinate system 3 14. 
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[0176] In the equation (9), the translation vector "T" indicates a direction of the origin point of 
the second coordination coordinate system 302 to the origin point of the camera coordination 
system 313 and a distance between the origin point of the second coordination coordinate system 
302 and the origin point of the camera coordination coordinate system 313. The rotation matrix 
"R" represents a deviation between the second coordination coordinate system 302 and the 
camera coordination coordinate system 313 in the rotational direction. When the housing unit 
31 1 is mounted on the slanted position on the automotive vehicle 308, the rotation matrix "R" is 
represented by the equations (3) to (5) shown in the first embodiment. Here, the legends 9, O, 
and 9 represent the angle of the camera coordination coordinate system 313 to the x-axis of the 
camera coordination coordinate system 313, the angle of the camera coordination c oordinate 
system 313 to the y-axis of the camera ooordmation coordinate system 313, and the angle of the 
camera coordination coordinate system 313 to the z-axis of the camera coordination coordinate 
system 313, respectively. 

[0177] When the housing unit 3 1 1 is mounted on the accurate position on the automotive vehicle 
308, the plane-projected image producing section 361 is adapted to produce a plane-projected 
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image shown in FIG. 22(b) from the image inforaiation shown in FIG. 22(a). In this plane- 
projected image, the lines 373 and 374 are in parallel relationship with each other by reason that 
the plane-projected image is the same as the image taken by the imaginary camera 370 located in 
parallel relationship with the road surface 302a. 

[01781 When the camera coordination coordinate system 313 is moved around the x-axis under 
the condition that the housing xmit 311 is downwardly slanted with respect to the automotive 
vehicle 308, the plane-projected image producing section 361 is adapted to produce a plane- 
projected image shown in FIG. 23(b) from the image information shown in FIG. 23(a). 
[0179] When the camera coordination coordinate system 313 is moved around the y-axis under 
the condition that the housing unit 311 is leftwardly panned with respect to the automotive 
vehicle 308, the plane-projected image producing section 361 is adapted to produce a plane- 
projected image shown in FIG. 24(b) from the image information shown in FIG. 24(a). 
[0180] When the camera coordinatio n coordinate system 313 is moved around the z-axis under 
the condition that the hoiising unit 311 is leftwardly slanted with respect to the automotive 
vehicle 308, the plane-projected image producing section 361 is adapted to produce a plane- 
projected image shown in FIG. 25(b) from the image information shown in FIG. 25(a), 
r01811 As shown in FIGS. 22(b), 23(b), 24(b), and 25(b), the plane-projected image dividing 
section 362 is adapted to divide the plane-projected image into a plurality of image segments on 
the basis of the image information of the dividing markers 307 obtained by the camera 310. 
[0182] The plane-projected image dividing section 362 is adapted to superimpose dividing lines 
383 and 384 on the plane-projected image on the basis of a base line 380 passing through the 
dividing markers 307. The dividing line 383 is in perpendicular relationship with the base line 
380, and passes thought an intermediate point 381 defined between the dividing markers 307. On 
the other hand, the dividing line 384 is in parallel and spaced relationship with the base line 380. 
r0183] When the housing unit 311 is mounted on the slanted position of the automotive vehicle 
308, more specifically, the camera coordination coordinate system 313 is moved around the z- 
axis, the base line 380 has a tilt to the right as will be seen from the plane-projected image shown 
in FIG. 25(b). 

r01841 As shown in FIGS. 22(b), 23(b), 24(b), and 25(b), the motion vector extracting section 
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363 is adapted to extract the motion vectors 388a, 388b, 388c, and 388d from the four image 
segments 386a, 386b, 386c, and 386d divided by the plane-projected image dividing section 362. 
r0185] The directions of the motion vectors 388a to 388d are eqxial to one another when the 
automotive vehicle 308 is linearly traveling by reason that the motion vectors 388a to 388d are 
calculated on the basis of the flow of the fragmental image of the image information obtained 
from the motion of the automotive vehicle 308, i.e., the motion of the camera 310 to the road. 
The following description will be directed to the motion vectors 388a to 388d extracted from the 
image information obtained under the automotive vehicle 308 is linearly traveling. 
[0186] When the housing unit 3 1 1 is mounted on the accurate position of the automotive vehicle 
308, the amplitudes of the motion vectors 388a, 388b, 388c, and 388d are equal to one another as 
shown in FIG. 22(b). 

[0187] When the housing unit 311 is mounted on the downwardly slanted position of the 
automotive vehicle 308, more specifically, the camera coordination coordinate system 313 is 
moved around the x-axis, the motion vectors 388a and 338b of the upper side fragmental image 
is larger than the motion vectors 388a and 338b of the lower side fragmental image as shown in 
FIG. 23(b). 

[0188] When the housing unit 311 is mounted on the leftwardly slanted position of the 
automotive vehicle 308, more specifically, the camera coordination coordinate system 313 is 
moved aroimd the y-axis, the motion vectors 388b and 338d of the right side fragmental image is 
larger than the motion vectors 388a and 338c of the left side fragmental image as shown in FIG. 
24(b). 

[0189] The calibration value calculating section 364 is adapted to produce a calibration value of 
the second optical position information on the basis of a following method. 
r0l901 As shown in FIGS. 22(c), 23(c), and 24(c), the vector 390a equal in amplitude to the 
motion vector 388a, the vector 390b equal in amplitude to the motion vector 388b, the vector 
390c equal in amplitude to the motion vector 388c, and the vector 390d equal in amplitude to the 
motion vector 388d are located on a composited vector calculating coordination coordinate 
system 391. 

[0191] Here, the vector 390a is located at the origin point of the composited vector calculating 
coordination coordinate system 391 in the direction of an upper-leftward angle of 45 degrees. 
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The vector 390b is located at the origin point of the composited vector calculating coordination 
coordinate system 391 in the direction of an upper-rightward angle of 45 degrees. The vector 
390c is located at the origin point of the composited vector calculating coordination coordinate 
system 391 in the direction of a lower-leftward angle of 45 degrees. The vector 390a is located at 
the origin point of the composited vector calculating coordination coordinate system 391 in the 
direction of a lower-rightward angle of 45 degrees. 

r01921 The composition of the vectors 390a to 390d is performed on the composited vector 
calculating coordination coordinate system 391 . 

[0193] When the housing unit 31 1 is mounted on the accurate position of the automotive vehicle 
308, the amplitude of the composited vector 392 is equal to zero by reason that the amplitudes of 
the motion vectors 388a, 388b, 388c, and 388d are equal to one another as shown in FIG. 22(b). 
Here, each of the rotation angle 9 to the x-axis, the rotation angle O to the y-axis, and the rotation 
angle (p to the z-axis is equal to zero on the camera coordination coordinate system 313. 
f0194] When the housing unit 311 is mounted on the downwardly slanted position of the 
automotive vehicle 308, and more specifically, the camera coordination coordinate system 313 is 
moved around the x-axis, the composited vector 392 points upward by reason that the vectors 
390a and 390b are respectively larger in amplitude than the vectors 390a and 390b as shown in 
FIG. 23(b). The rotation angle 9 to the x-axis is calculated on the basis of the equation (9) under 
the condition that the calculated rotation matrix "R" leads to the fact that the composited vector 
392 is equal to zero. 

[0195] When the housing unit 311 is moxmted on the leftwardly slanted position of the 
automotive vehicle 308, and more specifically, the camera coordination coordinate system 313 is 
moved around the y-axis, the composited vector 392 points rightward by reason that the vectors 
390b and 390d are respectively larger in amplitude than the vectors 390a and 390c as shown in 
FIG. 24(b). The rotation angle Q> to the y-axis is calculated on the basis of the equation (9) under 
the condition that the calculated rotation matrix "R" leads to the fact that the composited vector 
392 is equal to zero. 

r0196] When, on the other hand, the housing unit 31 1 is mounted on the slanted position of the 
automotive vehicle 308, and more specifically, the camera coordination coordinate system 313 is 
moved around the z-axis, the rotation angle q) to the z-axis of the camera coordination coordinate 
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system 313 is calculated on the basis of the slope of the base line 380 on the image coordination 
coordinate system 371 of the plane-projected image shown in FIG. 25(b). 
[0197] As shown in FIG. 26, the camera calibrating apparatus 300 thus constructed as previously 
mentioned can be realized by comprising a computer 391 for adjusting the camera 310, ECU 392 
constituted as an image controlling apparatus for controlling the camera 310, and other elements. 
r01981 The computer 391 is shown in FIG. 26(a) as includmg CPU, RAM, ROM, an input/output 
interface, and other units. The computer 391 is electrically connected to the camera 310 at the 
first working space. In this embodiment, the computer 391 constitutes each of the first housing 
position information storing section 315, the second housing position information storing section 
316, the first optical position information producing section 317, the fu-st optical position 
information storing section 318, the second optical position information producing section 320, 
the revising marker position information storing section 325, and the second optical position 
information storing section 330. 

101991 The ECU 392 is shown in FIG. 26(c) as includmg CPU, RAM, ROM, an input/output 
interface, and other elements. The ECU 392 is mounted on the automotive vehicle 308 to be 
electrically connected to the camera 310 in the second working space. In this embodiment, the 
ECU 392 constitutes each of the second optical position information storing section 330 and the 
calibrating section 360. 

[0200] As shown in FIG. 26(b), the camera 310 is transferred to the second working space firom 
the first working space witii a recordable medium 393 such as for example CD-ROM and a 
magnetic disk. The recordable medium 393 has stored therein the second optical position 
information to ensure that the second optical position information is effectively transferred to the 
ECU 392 fi-om tiie computer 391. 

r0201] In this embodiment, the camera 310 and the recordable medium 393 are transferred to the 
second working space from the first working space, however, the camera 310, the second optical 
position information storing section 330, and the calibrating section 360 may be collectively 
transferred as a camera unit 194 to the second working space from the first working space as 
shown in FIG. 27. 

[0202] The following description will be directed to the operation of the third embodiment of the 
camera calibrating apparatus according to the present invention. 
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[02031. The second optical position information is calibrated by the camera calibrating apparatus 
300 through the steps shown in the FIG. 28. 

[02041 The camera 310 is firstly located at the predetermined position on the first coordination 
coordinate system 301 in the first working space (in the step S301). The first housing position 
information, the second housing position information, and the revising marker position 
information are then stored in the furst housing position information storing section 315, the 
second housing position information storing section 316, and the revising marker position 
information storing section 325, respectively (in the step S302). Here, the first housing position 
information, the second housing position information, and the revising marker position 
information are obtained on the basis of positions predetermined in the design process and 
measured by a measuring instrument. 

r02051 The image mformation of the revising marker 105 is then obtained by the camera 310 (in 
the step S303). The fu-st optical position information is then produced by the first optical position 
information producing section 317 fi-om the revising marker position information stored in the 
revising marker position information storing section 325 on the basis of the image information of 
the revising marker 305 obtained by the camera 310 (in the step S304). The first optical position 
information produced by the fu:st optical position information producing section 317 is then 
stored in the first optical position information storing section 3 1 8 (in the step S305). 
[0206] The second optical position information is then produced by the second optical position 
information producing section 320 firom the second housing position information stored in the 
second housing position information storing section 316 on the basis of the first housing position 
information stored in the first housing position information storing section 315 and the first 
optical position information stored in the first optical position information storing section 318 (in 
the step S306). The second optical position information produced by the second optical position 
information producing section 320 is then stored in the second optical position information 
storing section 330 (in the step 8307). 

r02071 The camera 310 and the recordable medium 393 are then transferred to the second 
working space fi-om the first working space. The camera 310 is then mounted to the automotive 
vehicle 308 to be located at a predetermined position on the second coordination coordinate 
system 302 (in the step S308). 
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r02081 The image of the road surface 302a is then taken by the camera 310 (in the step 8309). As 
shown in FIGS. 22(b), 23(b), 24(b), and 25(b), the plane-projected image is produced from the 
image information obtained by the camera 310 on the second coordination coordinate system 
302 (in the step 8310). 

r02091 As shown m FIGS. 22(b), 23(b), 24(b), and 25(b), the plane-projected image produced by 
the plane-projected image producing section 361 is divided into a plurality of image segments 
386a to 386d by the plane-projected iniage dividing section 362 on the basis of the image 
information of the dividing markers 307 obtained by the camera 310 (in the step S311). 
[02101 As shown in FIGS. 22(b), 23(b), 24(b), and 25(b), the motion vectors are extracted by the 
motion vector extracting section 363 from the image segments 386a to 386d divided by the 
plane-projected image dividing section 362 (in the step S3 12). 

r02111 As shown in FIGS. 25(b), the calibration value on z-axis of the second optical position 
information stored m the second optical position information storing section 330 is then 
calculated by the calibration value calculating section 365 on the basis of the slope of the base 
line 380 on the image coordination coordinate system 371 of the plane-projected image shown in 
FIG. 25(b) (in the step S3 13). As shown in FIGS. 22(c), 23(c), and 24(c), the calibration value on 
each of x-axis and y-axis of the second optical position information stored in the second optical 
position information storing section 330 is then calculated by the calibration value calculating 
section 365 on the basis of the motion vectors extracted by the motion vector extracting section 
363(mthestepS314). 

[02121 The second optical position information stored in the second optical position information 
storing section 330 is finally calibrated by the optical position information calibrating section 
365 on the basis of the calibration value calculated by the calibration value calculating section 
364 (in the step S315). In this embodiment, the above mentioned steps 301 to 315 may be 
executed as a program by a computer. 

[02131 From the above detailed description, it will be understood that the camera calibrating 
apparatus 300 can calibrate the second optical position information by using the motion vectors 
388ato388d. 

[0214] In this embodiment, the camera calibrating apparatus 300 can easily extract the motion 
vectors from the respective image segments 386a to 386d. 
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r02151 In this embodiment, the camera calibrating apparatus 300 can accurately divide the plane- 
projected image by using the dividing marker 307. 
r02161 (Fourth Embodiment) 

[02171 Referring now to the drawings, in particular to FIGS. 29 to 38, there is shown a fourth 
embodiment of a camera calibrating apparatus according to the present invention. 
[0218] The construction of the fourth embodiment of the camera calibrating apparatus according 
to the present invention will now be described hereinafter. 

r02191 As shown in FIGS. 29 to 31, the camera calibrating apparatus 400 is electrically 
coimected to a camera 410 as an imaging device. The camera 410 includes a housing unit 41 1 
and an optical section 412 supported by the housing unit 411 to obtain image information 
through the optical section 412. 

[02201 Th^ camera calibrating apparatus 400 comprises a first housing position information 
storing section 415 for storing first housing position information indicative of the position of the 
housing imit 411 to the first coordination coordinate system 401, a second housing position 
information storing section 416 for storing second housing position information indicative of the 
position of the housing unit 411 in the second coordination coordinate system 402, a revising 
marker position information storing section 425 for storing revising marker position information 
indicative of the position of a revising marker 405 to the first coordination coordinate system 
401, and a vehicle position information storing section 426 for storing vehicle position 
information indicative of the position of, for example, a rear bumper 409 of an automotive 
vehicle 408 in the second coordination coordinate system 402. 

r0221] The first coordination coordinate system 401 is defined in a first working space such as a 
camera production plant to have Xi-axis, Yi-axis, and Zi-axis. The revising marker 405 laid out 
in the first working space causes the camera calibrating apparatus 400 is revise the camera 410 
located in the first working space. The revising marker 405 is constituted by a plurality of 
marking objects which are respectively located at predetermined positions on the first 
coordination coordinate system 401, and which are within a viewing field of the camera 410 
located in the first working space. 

r0222] The second coordination coordinate system 402 is defined in a second working space 
such as an automotive vehicle production plant to have X2-axis, Yi-axis, and Z2-axis, while X2- 
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axis and Y2-axis collectively define X2-Y2 plane which is represented by a road surface 402a on 
which an automotive vehicle 408 is located. The calibrating plate 406 painted in a single color is 
located on the road surface 402a, The monochromatic calibrating plate 406 located directly 
below the automotive vehicle 408 is different in brightness, chromaticity, and saturation of color 
from the automotive vehicle 408. 

[0223] The camera calibrating apparatus 400 is adapted to revise the camera 410 in the first 
working space. The camera 410 is located at a predetermined position on the first coordination 
coordinate system 401, while the first housing position information storing section 415 is 
adapted to store first housing position information indicative of the position of the housing unit 
41 1. Here, the term "the calibration of the camera 410" is intended to indicate the calculation of 
the accurate position of the optical section 412 of the camera 410 mounted on the automotive 
vehicle 408. 

r0224] The camera 410 revised by the camera calibrating apparatus 410 is mounted on the 
automotive vehicle 408 to be located at a predetermined position on the second coordination 
coordinate system 402 in the second working space, while the second housmg position 
information storing section 416 is adapted to store second housing position information 
indicative of the position of the housing unit 411. Here, the second housing position information 
indicates an accurate position in which the housing unit 411 of the camera 410 is located when 
the camera 410 is mounted on the automotive vehicle 408. 

f02251 The camera calibrating apparatus 400 further comprises a first optical position 
information producing section 417 for producing fu-st optical position information indicative of 
the position of the optical section 412 to the first coordination coordinate system 401, and a first 
optical position information storing section 418 for storing the first optical position information 
produced by the first optical position information producing section 417. 
r0226] The first optical position information producing section 417 is adapted to calculate the 
position of the optical section 412 to the furst coordination coordinate system 401 from the 
revising marker position information stored in the revising marker position information storing 
section 425 on the basis of the image information of the revising marker 305 obtained by the 
camera 410. Here, the term "the position of the optical section 412" is intended to indicate an 
optical center of the optical section 412 and the position of an optical axis, and other optical 
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parameters. The calculation of the position of the optical section 412 to the first coordination 
coordinate system 401 is performed on the basis of the method disclosed in the document 1 . 
r02271 The camera calibrating apparatus 400 further comprises a second optical position 
information producing section 420 for producing second optical information indicative of the 
position of the optical section 412 in the second coordination coordinate system 402, and a 
second optical position information storing section 430 for storing the second optical position 
information produced by the second optical position information producing section 420. 
r02281 The second optical position information producing section 420 is adapted to produce 
second optical position information indicative of the position of the optical section 412 in the 
second coordination coordinate system 402 from the second housing position information stored 
in the second housing position information storing section 416 on the basis of the first housing 
position information stored by the first housing position information storing section 415 and the 
first optical position information stored m the first optical position information storing section 
418. 

r02291 The second optical position information producing section 420 is adapted to calculate the 
position of the optical section 412 in the second coordination coordinate system 402 on the basis 
of a method which is the same as the method of the second optical position information 
producing section 120 of the first embodiment. 

r02301 As shown in FIG. 32, the camera 410 mounted on the automotive vehicle 408 in the 
second working space defines a camera coordination coordinate system 413 having x-axis, y- 
axis, and z-axis on the basis of the second optical position information. The origin point of the 
camera coordination system 413 is in register with the optical center of the optical section 412. 
The X-axis of the camera coordination coordinate system 413 extends in a horizontal direction of 
the camera 410. The y-axis of the camera coordination coordinate system 413 extends in a 
vertical direction of the camera 410. The z-axis of the camera coordination coordinate system 
413 is in axial alignment with the optical axis of the optical section 412. 
[0231] The image coordination coordinate system 414 is defined on a plane spaced apart firom 
the origin point of the camera coordination coordinate system 413 with a focal length "f in the 
direction of the z-axis of the camera coordination coordinate system 413. The image 
coordination system 414 has p-axis and q-axis. The camera 410 is adapted to obtain, as image 
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information, an optical image focused on the image ooordination coordinate system 414 through 
the optical section 412. 

r02321 The camera calibrating apparatus 400 further comprises an estimated location position 
information producing section 440 for producing estimated location position information of the 
bumper 409 of the automotive vehicle 408 to the image coordination coordin ate system 414 of 
the cameira 410, and an estimated location position information storing section 450 for storing 
the estimated location position information produced by the estimated location position 
information producing section 440. 

[02331 Th^ estimated location position information producing section 440 is adapted to produce 
estimated location position information of the calibrating marker 406 to the image coordination 
coordinate system 414 of the camera 410 from the position information of the calibrating marker 
406 stored in the vehicle position information storing section 426 on the basis of the second 
optical position information produced by the second optical position information producing 
section 420. The calculation of the estimated location of the calibrating marker 406 to the image 
ooordination coordinate system 414 of the camera 410 is performed on the basis of the above 
mentioned document 1 . 

r0234] As shown in FIG. 33, the bumper 409 of the automotive vehicle 408 located on the 
second ooordination coordinate system 402 is focused on the image location Pn* of the im£^e 
ooordination coordinate system 414 through the optical section 412. Here, the image location Pn* 
is in register with the estimated image location Pn calculated by the estimated image location 
calculating section 440 under the condition that the housing unit 411 is located on an accurate 
position of the automotive vehicle 408, more specifically, the housing unit 41 1 is located at the 
position represented by the second housing position information with no deviation to the 
accurate position. However, the housing unit 41 1, in general, is located at an inaccurate position 
with deviation to the accurate position. This leads to the fact that the second optical position 
information is produced with deviation to the accurate position. As a result, the estimated 
location Pn is spaced apart from the image location Pn' of the image ooordination coordinate 
system 414. 

[0235] The camera calibrating apparatus 400 further comprises a calibrating section 460 for 
calibrating the second optical position information stored in the second optical position 
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information storing section 430 to order to calibrate the above mentioned deviation of the second 
optical position information. 

[0236] The calibrating section 460 is adapted to calibrate the second optical position information 
stored in the second optical position information storing section 430 on the basis of the image 
information of the bximper 409 obtained by the camera 410 and the estimated location position 
information stored in the estimated location position information storing section 450. 
[02371 The calibrating section 460 includes an image location information extracting section 470 
for extracting image location information indicative of an image location of the bumper 409 to 
the unage coordination coordinate system 414 of the camera 410 on the basis of the image 
information of the calibrating marker 406 obtained by the camera 110, a calibration value 
calculating section 480 for calculating a calibration value of the second optical position 
information stored in the second optical position information storing section 430 on the basis of 
the image location information extracted by the image location information extracting section 
470 and the estimated location position information stored in the estimated location position 
information storing section 450, and an optical position information calibrating section 490 for 
calibrating the second optical position information stored in the second optical position 
information storing section 430 on the basis of the calibration value calculated by the calibration 
value calculating section 480. 

[02381 The calibration value calculating section 480 includes a superimposing section 481 for 
superimposing one of a profile line of the bumper 409 of the image location information and a 
profile line of the bumper 409 of the estimated location information on the other of the profile 
line of tiie bumper 409 of the image location information and the profile line of the bumper 409 
of the estimated location information, an extracting section 482 for extracting two or more 
points, for example points of end portions of the bumper, from the profile lines of the automotive 
vehicle superimposed by the superimposing section 481, and a calculating section 483 for 
calculating the calibration value of the second optical position information stored by the second 
optical position information by comparing the points of the image location information with the 
points of the estimated location information. 

[0239] The superimposing section 481 is adapted to superimpose the profile line " P " (see FIG. 
34(a)) of the bumper 409 of the estimated location information on the profile line" P' " (see FIG. 
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34(b)) of the bumper 409 of the image location information by allowing the profile line " P " of 
the bumper 409 of the estimated location information to be linearly and rotationally moved with 
respect to the profile line " P' " of the bumper 409 of the unage location information (see FIG. 
34(c)). 

r0240] As shown in FIG. 35(a), the extracting section 482 is adapted to extract two or more 
points (Pi, P2) and (Pi', P2') firom the profile Imes " P " and " P* " of the bumper 409 
superimposed by the superimposing section 481 . 

[0241] The calculating section 483 is adapted to calculate a calibration value of the second 
optical position information on the basis of a method which is the same as the method of the 
calibration value calculating section 180 in the first embodiment by comparing the points (Pi, P2) 
(see FIG. 35(b)) of the estimated location information with the points (Pi', P2') (see FIG. 35(b)) 
of the image location information. 

[0242] As shown in FIG. 36, the camera calibrating apparatus 400 thus constructed as previously 
mentioned can be realized by comprising a computer 491 for adjusting the camera 410, ECU 492 
constituted as an image controlling apparatus for controlling the camera 410, and other elements. 
r02431 The computer 491 is shown in FIG. 26(a) as mcluding CPU, RAM, ROM, an input/output 
interface, and other elements. The computer 491 is electrically connected to the camera 410 at 
the furst working space. In this embodiment, the computer 491 constitutes each of the fu^t 
housing position information storing section 415, the second housing position information 
storing section 416, the first optical position information producing section 417, the first optical 
position information storing section 418, the second optical position information producing 
section 420, the calibrating marker position information storing section 425, the calibrating 
marker position information storing section 426, the second optical position storing section 430, 
the estimated location p osition information producing section 440, and the estimated location 
position information storing section 450. 

r02441 The ECU 492 is shown in FIG. 36(c) as mcludmg CPU, RAM, ROM, an input/output 
interface, and other elements. The ECU 492 is mounted on the automotive vehicle 408 to be 
electrically connected to the camera 410 in the second working space. In this embodiment, the 
ECU 492 constitutes each of the second optical position information storing section 430, the 
estimated location position information storing section 450, and the calibrating section 460. 
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r02451 As shown in FIG. 36(b), the camera 410 is transferred to the second working space from 
the first working space with a recordable medium 493 such as for example CD-ROM and a 
magnetic disk. The recordable medium 493 has stored therein each of the second optical position 
information and the estimated location information to ensure that the each of the second optical 
position information and the estimated location information is effectively transferred to the ECU 
492 from the computer 49 1 . 

r0246] In this embodiment, the camera 410 and the recordable medium 493 are transferred to the 
second working space from the first working space, however, the camera 410, the second optical 
position information storing section 430, and the calibrating section 460 may be collectively 
transferred as a camera unit 494 to the second working space from the first working space as 
shown in FIG. 37. 

r0247] The following description will be directed to the operation of the fourth embodiment of 
the camera calibrating apparatus according to the present invention. 

r02481 The second optical position information is calibrated by the camera calibrating apparatus 
400 through the steps shown in the FIG. 38. 

[0249] The camera 410 is firstly located at the predetermined position on the fu-st coordination 
coordinate system 401 in the first working space (in the step 840 1). The first housmg position 
information, the second housing position information, the revising marker position information, 
and the vehicle position information are then stored in the first housing position information 
storing section 415, the second housing position information storing section 416, the revising 
marker position information storing section 425, and the vehicle position information storing 
section 426, respectively (in the step S402). Here, the first housmg position information, the 
second housing position information, the revising marker position information, and the vehicle 
position information are obtained on the basis of positions predetermined in the design process 
and measured by a measuring instrument. 

r0250] The image information of the revising marker 405 is then obtained by the camera 410 (in 
the step 8403). The furst optical position mformation is then produced by the first optical position 
mformation producing section 417 from the revising marker position information stored in the 
revising marker position information storing section 425 on the basis of the image information of 
the revising marker 405 obtained by the camera 410 (in the step 8404). The fu'st optical position 
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information produced by the first optical position information producing section 417 is then 
stored in the fu*st optical position information storing section 418 (in the step S405). 
r025n The second optical position information is then produced by the second optical position 
information producing section 420 from the second housing position information stored in the 
second housing position information storing section 416 on the basis of the first housing position 
information stored in the first housing position information storing section 415 and the first 
optical position information stored in the first optical position information storing section 41 8 (in 
the step S406). The second optical position information produced by the second optical position 
information producing section 420 is then stored in the second optical position information 
storing section 430 (in the step S407). 

r0252] The estimated location information indicative of the position of the bumper 409 to the 
image coordination coordinate system 414 of the camera 410 is then produced by the estimated 
location position information producing section 440 fi-om the vehicle position information stored 
in the vehicle position information storing section 426 on the basis of the second optical position 
information produced by the second optical position information producing section 420 (in the 
step 8408). The estimated location position information produced by the estimated location 
position information producing section 440 is stored in the estimated location position 
information storing section 450 (in the step S409). 

[0253] The camera 410 and the recordable medium 493 are then transferred to the second 
working space from the first working space. The camera 410 is then moxmted to the automotive 
vehicle 408 to be located at the predetermined position on the second coordination coordinate 
system 402 (m tiie step S410). 

r0254] The image of the bumper 409 is then taken by the camera 310 against the calibrating plate 
406 (in the step 841 1). As shown in FIG. 33, the image location information of the bumper 409 
is extracted from the image information of the bumper 409 obtained by the camera 410 on the 
image coordination coordinate system 414 of the camera 410 (in the step 8412). 
[0255] As shown in FIG. 34, the profile line P' of the bumper 409 of the image location 
information is superimposed on the profile line P of the bumper 409 of the estunated image 
information by the superimposing section 481 (in the step 8413). 
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[0256] As shown in FIG. 35, two or more points (Pi, P2) and (Pi', P2O of the overlapped portion 
are then extracted from the profile lines " P " and " P* " of the bumper 409 by the extracting 
section 482 (in the step S414). The calibration value of the second optical position information is 
then calculated by the calculating section 483 by comparing the points (Pi', P2') of the image 
location information with the points (Pi, P2) of the estimated location information (in the step 
841 5). 

|'0257] The second optical position information stored in the second optical position inforaiation 
storing section 430 is finally calibrated by the optical position information calibrating section 
490 on the basis of the calibration value calculated by the calibration value calculating section 
480 (in the step S416). In this embodiment, the above mentioned steps 401 to 415 may be 
executed as a program by a computer. 

[0258] From the above detailed description, it will be understood that the camera calibrating 
apparatus 400 can calibrate Ihe second optical position information by using the motion vectors 
388ato388d. 

[02591 In this embodiment, the camera calibrating apparatus 400 can calibrate the second optical 
position information by using the bumper 409 forming part of the automotive vehicle 408. 
[0260] In this embodiment, the camera calibrating apparatus 400 can accurately divide the plane- 
projected image by using the dividing marker 307. 

[0261] As will be seen from the above detailed description, the camera calibrating apparatus 
according to the present invention can calibrate the optical parameter of the camera mounted to 
the automotive vehicle. 
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